Facial profile password to modify user account data for hands free transactions

ABSTRACT

An account management system establishes an account for a user. The user enters user account information into the account and the account management system establishes a facial template for the user based on an image of the face of the user. The user requests to change user account information at a merchant POS (POS) device. The merchant POS device captures a facial image of the user and transmits the image the account management system, which generates a facial template and compares the generated facial template against the existing facial template associated with user account. If the generated facial template is less than a threshold difference from the existing facial template, the user may update user account information at the merchant POS device, which communicates the updated user account information to the account management system. The account management system associates the updated user account information with the user account.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to and is a continuation of U.S. patentapplication Ser. No. 15/143,453, filed Apr. 29, 2016, and entitled“Facial Profile Password to Modify User Account Data for Hands-FreeTransactions,” which claims priority to U.S. Patent Application No.62/155,466, filed Apr. 30, 2015, and entitled “Identifying Consumers viaFacial Recognition to Provide Services,” and claims priority to U.S.Patent Application No. 62/302,142, filed Mar. 1, 2016, and entitled“Facial Profile Modification for Hands Free Transactions.” The completedisclosure of the above-identified priority applications is hereby fullyincorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to improving user security for updates touser account information by requiring updated user biometric informationassociated with user payment accounts to substantially agree withpreviously submitted biometric information.

BACKGROUND

When consumers make purchases at a merchant location, many methods ofconducting a transaction are available. Consumers may use many differentcards or accounts for purchases, such as gift cards, debit cards, creditcards, stored value cards, and other cards or accounts. The user accountidentifiers and other data represented by the cards may be communicatedto the merchant system via magnetic stripes, near field communicationtechnologies involving user computing devices, and other suitablemechanisms.

Current applications for conducting transactions at a merchant locationmay provide the opportunity for the consumer to make a hands-freetransaction verified via biometric information of a user, such as imageor voice recognition of a user at checkout. However, currentapplications may not adequately prevent illegitimate users from updatingbiometric information of a user.

SUMMARY

Techniques herein provide computer-implemented methods to processhands-free transactions with facial recognition of a user and forenabling the user to update account information for use in hands-freetransactions via a password that comprises a facial template generatedfrom an image of the user that was captured at the time the user desiresto update the account information. In an example embodiment, a merchantsystem registers with an account management system. The merchant systeminstalls one or more merchant beacon devices and one or more merchantPOS devices at a merchant location. A user establishes an account withthe account management system and downloads an application on a usercomputing device associated with the user. In an example, the usertransmits an image of himself and/or an audio recording of himself tothe account management system to establish a facial template and/oraudio template associated with the user account. The user may alsoconfigure, via the user computing device, a challenge and response toassociate with the user account, and the account management systemreceives the configuration of the challenge and response via the usercomputing device. For example, the challenge and response may comprise aquestion and answer. In this example, the question may read “What isyour mother's maiden name?” and the answer may read “Smith.” In anotherexample, the challenge and response may comprise a request for apassword and a password.

After initially configuring the facial template audio template, and/orchallenge and response for the user account, the user desires to updatethe account information associated with the user account. This update ofinformation can be performed via a computing device selected by theuser, such as a merchant POS device, the user computing device, or viaanother appropriate computing device able to capture a facial image ofthe user or communicatively coupled to a computing device able tocapture a facial image of the user. The selected computing devicedisplays a request for the user to take a facial picture of himself. Inanother example, the selected computing device activates an audiocomponent and displays a request for the user to record audio of himselfvia the computing device. The selected computing device transmits therecorded audio or facial image to the account management system, whichgenerates a facial template or audio template based on the capturedimage and/or audio from the user. In yet another example, the selectedcomputing device displays the challenge and a request to provide theresponse as input. The selected computing device receives an input ofthe response and transmits the response to the account managementsystem.

The account management system compares the generated facial template oraudio template against the facial template or audio template associatedwith the user account. If the difference between the generated facial oraudio template and the corresponding template associated with the useraccount is greater than or equal to a threshold amount, the accountmanagement system alerts the selected computing device that the facialtemplate or audio template was not successfully matched to the useraccount and that the user may not update user account information. Inanother example, if a similarity between the generated facial or audiotemplate and the corresponding template associated with the user accountis less than the threshold amount, the account management system alertsthe selected computing device that the facial template or audio templatewas not successfully matched to the user account and that the user maynot update user account information. If the difference between thegenerated facial or audio template and the corresponding templateassociated with the user account is less than or equal to a thresholdamount, the account management system transmits a notification to theselected computing device that the facial template or audio template wassuccessfully matched to the user account and the user may update useraccount information. In another example, if a similarity between thegenerated facial or audio template and the corresponding templateassociated with the user account is greater than a threshold amount, theaccount management system transmits a notification to the selectedcomputing device that the facial template or audio template wassuccessfully matched to the user account and the user may update useraccount information. In another example, in addition to or instead ofcomparing a generated facial template and/or audio template against acorresponding template associated with the user account, the accountmanagement system compares the received response to the challengeagainst a response associated with a challenge associated with the useraccount. In this example, if the response received from the selectedcomputing device matches the response associated with the challenge inthe user account, the account management system transmits a notificationto the selected computing device that the user may update user accountinformation.

In response to receiving a notification from the account managementsystem that the facial template or audio template was successfullymatched to the user account and/or that the user may update accountinformation, the selected computing device displays an interface bywhich a merchant POS device operator or the user may enter updated useraccount information. In this example, the user enters the updatedaccount information and the selected computing device transmits theupdated account information to the account management system. Updatingaccount information may comprise adding information associated with oneor more new payment accounts, editing information associated with one ormore existing payment accounts, deleting data associated with one ormore existing payment accounts, adding personal data, deleting personaldata, and/or editing personal data. In this example, personal data maycomprise one or more of contact information of the user, demographicinformation of the user, one or more preferences of the user, one ormore permissions granting permission to the account management system toconduct one or more actions with respect to hands free transactions,and/or other relevant personal data. In another example embodiment,updating account information may comprise updating the facial template,audio template, or challenge and response associated with the useraccount. If successfully updated, the user may engage in hands-freetransactions using the updated account information associated with theuser account according to the example embodiments described herein.

In certain other example aspects described herein, systems and computerprogram products to process hands-free transactions with facialrecognition of a user and for enabling the user to update accountinformation for use in hands-free transactions via a password comprisinga facial template generated based on an image of the user captured atthe time the user desires to update the account information areprovided.

These and other aspects, objects, features, and advantages of theexample embodiments will become apparent to those having ordinary skillin the art upon consideration of the following detailed description ofillustrated example embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram depicting a system for processing hands-freetransactions with facial recognition of a user and for enabling the userto update user account information for use in hands-free transactionsvia a password comprising a facial template, in accordance with certainexample embodiments.

FIG. 2 is a block flow diagram depicting a method for processinghands-free transactions with facial recognition of a user and forenabling the user to update user account information for use inhands-free transactions via a password comprising a facial template, inaccordance with certain example embodiments.

FIG. 3 is a block flow diagram depicting a method for registering, by amerchant system, with an account management system and installinghardware at a merchant location, in accordance with certain exampleembodiments.

FIG. 4 is a block flow diagram depicting a method for registering, by auser, for an account with an account management system, in accordancewith certain example embodiments.

FIG. 5 is a block flow diagram depicting a method for establishing afacial template associated with a user account, in accordance withcertain example embodiments.

FIG. 6 is a block flow diagram depicting a method for establishing anaudio template associated with a user account, in accordance withcertain example embodiments.

FIG. 7 is a block flow diagram depicting a method for receiving, by auser computing device, a merchant beacon identifier broadcasted by amerchant beacon device, in accordance with certain example embodiments.

FIG. 8 is a block flow diagram depicting a method for receiving, by amerchant POS device, a facial template and a payment token for each userin range of a merchant beacon device, in accordance with certain exampleembodiments.

FIG. 9 is a block flow diagram depicting a method for updating, by amerchant POS device, a customer log as users enter or leave a networkrange of a merchant beacon device, in accordance with certain exampleembodiments.

FIG. 10 is a block flow diagram depicting a method for initiating, by auser, a transaction at a merchant POS device, in accordance with certainexample embodiments.

FIG. 11 is a block flow diagram depicting a method for identifying, by amerchant POS device, a user via facial recognition, in accordance withcertain example embodiments.

FIG. 12 is a block flow diagram depicting a method for identifying, by amerchant POS device, a user via voice recognition, in accordance withcertain example embodiments.

FIG. 13 is a block flow diagram depicting a method for identifying, by amerchant POS device operator, a user via a challenge and a response, inaccordance with certain example embodiments.

FIG. 14 is a block flow diagram depicting a method for conducting atransaction, in accordance with certain example embodiments.

FIG. 15 is a block flow diagram depicting a method for updating, by auser, user account information at a merchant POS device via a facialtemplate password, in accordance with certain example embodiments.

FIG. 16 is a block diagram depicting a computing machine and module, inaccordance with certain example embodiments.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS Overview

The example embodiments described herein provide computer-implementedtechniques for processing hands-free transactions with facialrecognition of a user and for enabling the user to update user accountinformation for use in hands-free transactions via a password comprisinga facial template.

In an example embodiment, a merchant system registers with an accountmanagement system. The merchant system installs one or more merchantbeacon devices and one or more merchant POS devices at a merchantlocation. A user establishes an account with the account managementsystem and downloads an application on a user computing deviceassociated with the user. In an example, the user transmits an image ofhimself and/or an audio recording of himself to the account managementsystem to establish a facial template and/or audio template associatedwith the user account. The user enters a merchant location and signsinto the application via the user computing device. The user computingdevice receives a merchant beacon device identifier broadcasted at themerchant location from the merchant beacon device and transmits themerchant beacon device identifier to the account management system. Theaccount management system transmits facial templates, audio templates,and/or challenges and responses to the merchant POS device associatedwith users whose user computing devices are within network range of themerchant beacon device and who are signed in to the application.Additionally, the account management system generates a payment tokenfor each user whose user computing device is within network range of themerchant beacon device and is signed into the application. An examplepayment token comprises a series of alphanumeric and/or symboliccharacters. The example payment token may be associated with a paymentaccount of the user and be recognizable by an issuer system associatedwith the payment account of the user. For example, the accountmanagement system generates the payment token and communicates thepayment token to an issuer system associated with a payment account ofthe user along with the user payment account information. In thisexample, if the issuer system, at a later time, receives the paymenttoken from a POS device in a payment transaction, the issuer system canextract the user payment account information associated with the paymenttoken.

A merchant camera device associated with the merchant POS devicecaptures a facial image of the user and the merchant POS deviceidentifies the user based on comparing the captured facial image againstthe received facial templates. Alternatively, the user submits an audiorecording to the merchant POS device, which identifies the user based oncomparing the received audio recording against audio templates forusers. In yet another embodiment, the merchant POS device operatoridentifies the user based on a user's response to a challenge. Afteridentifying the user, the merchant POS device processes a transactionusing the payment token associated with the user received from theaccount management system. The merchant POS device generates atransaction authorization request comprising the payment token andtransaction details, and transmits the transaction authorization requestto an issuer system associated with the user account selected for use inthe transaction. The issuer system identifies the user payment accountbased on the received payment token and processes the transaction usingthe transaction details and the user payment account information. Themerchant POS device receives an approval of the transactionauthorization request and transmits a receipt to the merchant POSdevice.

The user associated with the user account, at a time after initiallyconfiguring the user account data, may wish to update the user accountdata. Example user account data may comprise one or more user paymentaccount information, user demographic information, user contactinformation, a user facial template, a user audio template, and/orchallenge and response for the user account. In another example, theuser computing device is stolen from the user and an illegitimate userattempts to reconfigure the user account data for the user account sothat the illegitimate user can engage in hands-free transactions usingthe user computing device of the user. In another example, theillegitimate user attempts to change user account data at a merchant POSdevice or other appropriate computing device. The user (or other,illegitimate user) or merchant system operator selects an option via theapplication of the user computing device, merchant POS device, or othercomputing device to change the user account data. The applicationactivates the camera of the respective computing device and displays arequest for the user to take a facial picture of himself. In anotherexample, the application on the merchant POS device, user computingdevice, or other selected computing device activates an audio componentof the respective computing device and displays a request for the userto record audio of himself. The application on the selected computingdevice transmits the recorded audio or facial image to the accountmanagement system, which generates a facial template or audio templatebased on the captured image and/or audio from the user. The accountmanagement system compares the generated facial template or audiotemplate against the facial template or audio template associated withthe user account. If the difference between the generated facial oraudio template and the corresponding template previously associated withthe user account is greater than a threshold amount, the accountmanagement system transmits a notification to the application on therespective computing device via the network that the facial template oraudio template was not successfully updated. If the difference betweenthe generated facial or audio template and the corresponding templatepreviously associated with the user account is equal to or less than athreshold amount, the account management system associates the generatedfacial or audio template with the user account and alerts theapplication on the respective computing device that the facial templateor audio template was successfully updated. If successfully updated, theuser may engage in hands-free transactions using the updated facialtemplate or updated audio template.

In an example embodiment, a merchant system registers with an accountmanagement system. A merchant system operator installs an application ona merchant POS device. In another example, the merchant system operatorinstalls the application on a plurality of merchant POS devices at amerchant location. For example, the application comprises a paymentapplication able to communicate via the network with the accountmanagement system and managed by the account management system. Amerchant beacon device receives a beacon identifier code from theaccount management system. For example, the merchant system operatorinstalls one or more merchant beacon devices at the merchant location.The merchant beacon device broadcasts the merchant beacon identifiercode via wireless communication at the merchant location. The merchantsystem operator installs a merchant camera device at the merchantlocation to correspond to a corresponding merchant POS device. Inanother example, a plurality of merchant camera devices are installed atthe merchant location, each merchant camera device corresponding to aparticular merchant POS device. In yet another example, a particularmerchant camera device may correspond to two or more particular merchantPOS devices.

In an example embodiment, the user registers with an account managementsystem. For example, the user accesses an account management systemwebsite via a user computing device associated with the user. The userregisters with the account management system and downloads anapplication onto the user computing device. For example, the applicationcomprises a payment application able to communicate via the network withthe account management system and managed by the account managementsystem. In an example embodiment, the account management systemestablishes a facial template associated with the user account. Forexample, the application displays a request for the user to capture afacial image via the user computing device. The user selects an optionto capture a facial image. The application activates a camera on theuser computing device and the users captures a facial image of himself.The account management system receives the facial image. The accountmanagement system creates a facial template associated with the useraccount based on the received facial image. The account managementsystem deletes the received facial image. In another example embodiment,the account management system establishes an audio template associatedwith the user account. For example, the application requests andreceives user audio via the user computing device. The applicationtransmits the received user audio to the account management system andthe account management system creates an audio template associated withthe user account based on the received audio of the voice of the user.The account management system deletes the received audio of the voice ofthe user. In yet another example embodiment, the account managementsystem establishes a challenge and response associated with the useraccount. For example, the application on the user computing devicedisplays a challenge, such as “user initials,” and requests a responsefrom the user. In this example, user John Doe may enter “J. D.” as theresponse to the challenge. In this example, the application transmitsthe entered response to the account management system, which associatesthe response with the challenge in the user account.

The user signs into the application on the user computing device. Theuser carries the user computing device within a threshold distance of amerchant beacon device at the merchant location. The user computingdevice receives a merchant beacon identifier broadcast by the merchantbeacon device, and transmits the received merchant beacon identifier anda user account identifier to the account management system. The accountmanagement system receives the merchant beacon identifier and the useraccount identifier. The account management system extracts a facialtemplate associated with the user account identifier and identifies amerchant POS device associated with the merchant beacon deviceidentifier. In another example, the account management system extractsan audio template associated with the user account identifier and/or achallenge and response associated with the user account identifier inaddition to or instead of extracting the a facial template associatedwith the user account identifier.

The account management system transmits a facial template of theidentified user to the merchant POS device associated with the merchantbeacon device identifier. For example, a facial template associated withthe identified user's account is transmitted to the merchant POS device.The merchant POS device receives the facial template of the user, audiotemplate of the user, and/or challenge and response associated with theuser.

Additionally, the account management system generates a payment tokenfor each user whose user computing device is within network range of themerchant beacon device and is signed in to the application. An examplepayment token comprises a series of alphanumeric and/or symboliccharacters. The example payment token may be associated with a paymentaccount of the user and be recognizable by an issuer system associatedwith the payment account of the user. For example, the accountmanagement system generates the payment token and communicates thepayment token to an issuer system associated with a payment account ofthe user along with the user payment account information. In thisexample, if the issuer system, at a later time, receives the paymenttoken from a POS device in a payment transaction, the issuer system isable to extract the user payment account information associated with thepayment token.

The merchant POS device associates the payment token, the facialtemplate of the user, the audio template of the user, and the challengeand response of the user in a customer log. The merchant POS deviceperiodically updates the customer log based on updates received from theaccount management system. For example, the account management systemtransmits a subsequent facial template, audio template, challenge andresponse, and payment token for a subsequent user that enters athreshold distance of a merchant beacon device required to establish awireless network connection while carrying a computing device via whichthe subsequent user is signed into the application. In this example, theaccount management system receives the merchant beacon device identifiertransmitted by the subsequent user computing device and generates apayment token for the subsequent user. The account management systemtransmits the facial template of the subsequent user, audio template ofthe subsequent user, challenge and response of the subsequent user, andthe generated payment token to the merchant POS device. In anotherexample, in response to detecting that the computing device associatedwith a particular user listed on the customer log is no longermaintaining a network connection with the merchant beacon device, is nolonger retransmitting the merchant beacon device identifier to theaccount management system, and/or is no longer signed into theapplication, the account management system transmits a notice to themerchant POS device that the particular user has left a merchantlocation. In this example, the merchant POS device deletes the user fromthe customer log. For example, deleting the user comprises deleting theuser account identifier, facial template, audio template, challenge andresponse, payment token, and/or any other data associated with theparticular user from the merchant POS device.

The user approaches the merchant POS device. The merchant POS deviceoperator totals items of the user for purchase. The merchant POS deviceoperator asks the user to select a payment option. The user directs themerchant POS device operator to initiate a transaction via theapplication. The merchant POS device operator selects an option on themerchant POS device to initiate a transaction using the application.

The merchant camera device, communicatively coupled to the merchant POSdevice, captures video of the user. For example, the user is positionedin front of the POS device and the merchant camera device is positionedto be able to capture a video of the user's face. In an exampleembodiment, the merchant camera device starts capturing video of theuser only when the merchant POS device receives an input from anoperator of the merchant POS device to identify the user. In anotherexample embodiment, the merchant camera device starts capturing videowhen the associated merchant POS device receives an indication from theaccount management system that a user computing device has established anetwork connection with the merchant beacon device and/or retransmittedthe merchant beacon device identifier to the account management system.In this example embodiment, the merchant camera device does not capturevideo when there are no users with associated user computing deviceswithin network range of the merchant beacon device. The merchant cameradevice extracts a facial image of the user from the captured video andgenerates a facial template from the captured facial image. The merchantcamera device deletes the captured video and extracted facial image andtransmits the facial template to the merchant POS device. In anotherexample, the merchant camera device transmits the facial image of theuser to the merchant POS device and the merchant POS device generatesthe facial template from the facial image.

The merchant POS device retrieves facial templates from the customerlog. For example, the customer log comprises a list of users andassociated facial templates for each of the users that have computingdevices with an established a network connection to the merchant beacondevice at the merchant location and/or have retransmitted the merchantbeacon device identifier to the account management system. In an exampleembodiment, the customer log comprises volatile or transient memory. Forexample, the customer log is not saved and user information is added ordeleted from the customer log as user computing devices associated withrespective users enter or leave a network range of the merchant beacondevice. The merchant POS device compares the generated facial templatefrom the extracted facial image to facial templates from the customerlog. The merchant POS device is able to identify the user if there is amatch between a facial template from the customer log and the generatedfacial template. The merchant POS device is unable to identify the userif there is no match between a facial template from the customer log andthe generated facial template. If the merchant POS device is able toidentify the user, the merchant POS device notifies the accountmanagement system of the identity of the user and the account managementsystem processes a transaction between the user and the merchant system.In an example embodiment, if the merchant POS device is able to identifythe user, but is unable to notify the account management system of theidentity of the user, the merchant POS device processes the transactionusing the received payment token associated with user account of theidentified user.

In an example embodiment, if the user cannot be identified based onfacial recognition, the merchant POS device identifies the user based onaudio recognition. In an example embodiment, if the account managementsystem identifies users based on audio recognition, the accountmanagement system retrieves audio templates corresponding to users fromthe customer log. The merchant POS device displays a request to recordan audio of the user via a user interface of the merchant POS device.The merchant POS device records a voice input of the user and comparesthe received voice input to the retrieved audio templates thatcorrespond to the users from the customer log. The merchant POS deviceis able to identify the user if there is a match between an audiotemplate from the customer log and the received voice input of the user.The merchant POS device is unable to identify the user if there is nomatch between an audio template from the customer log and the receivedvoice input of the user. If the merchant POS device is able to identifythe user, the merchant POS device notifies the account management systemof the identity of the user and the account management system processesthe transaction between the user and the merchant system. In an exampleembodiment, if the merchant POS device is able to identify the user, butunable to notify the account management system of the identity of theuser, the merchant POS device processes a transaction using the receivedpayment token associated with user account of the identified user.

If the merchant POS device is unable to identify the user based onfacial and/or voice recognition, the merchant POS device operator isnotified, via a display on the merchant POS device, to issue a challengeto the user. The user provides a challenge response and the merchant POSoperator inputs the response into the merchant POS device. The merchantPOS device displays potential users from the customer log based on thechallenge response. For example, the merchant POS device accesses thecustomer log comprising a list or table that associates each challengewith corresponding the response, the user account identifiers, and thepayment tokens. In this example, the merchant POS device identifies theuser by correlating the challenge and the response to identify one ormore users in the customer log. In this example, the merchant POS devicedisplays the one or more identified users to the merchant POS deviceoperator. The merchant POS device operator selects a user. In anexample, the merchant POS device operator may compare a visual image orname of the user displayed on the user computing device to the visualappearance of the current customer at the merchant POS device and/ordocumentation presented by the user to the merchant POS operator. In anexample, the merchant POS device transmits the identity of the useridentified by the merchant POS operator. If the merchant POS deviceoperator is able to identify the user via the challenge and response,the merchant POS device notifies the account management system of theidentity of the user and the account management system processes thetransaction between the user and the merchant system. If the merchantPOS device operator is unable to identify the user via the challenge andresponse, the merchant POS device operator cancels the transaction byactuating one or more objects on the user interface of the merchant POSdevice. In an example embodiment, if the merchant POS device is able toidentify the user, but unable to notify the account management system ofthe identity of the user, the merchant POS device processes atransaction using the received payment token associated with useraccount of the identified user.

The merchant POS device operator confirms the transaction withpermission of the user. In an example embodiment, the merchant POSdevice generates a transaction authorization request based ontransaction details and the payment token associated with the userretrieved from the customer log. For example, transaction details maycomprise a total amount of the transaction, a selected user account foruse in the transaction, an account of the merchant for use in thetransaction, and other useful or relevant information. The merchant POSdevice transmits a transaction authorization request to an issuersystem. For example, the issuer system is associated with a user paymentaccount selected for use by the user in all hands free transactionsinvolving the application. The issuer system approves or denies thetransaction authorization request and transmits a transactionauthorization approval or denial of transaction authorization request tothe merchant POS device. The merchant POS device transmits a transactionreceipt to the user computing device and/or prints or displays a receiptfor the user at the merchant POS device indicating a status of thetransaction. For example, the merchant POS device displays an indicationthat the transaction was successfully processed or that the transactionwas denied.

In another example embodiment, the merchant POS device transmits anindication of an identity of the user identified via facial, audio,and/or challenge and response to the account management system alongwith the transaction details. In this example embodiment, the accountmanagement system processes the transaction with the issuer system. Forexample, the account management system generates a transactionauthorization request comprising the payment token, where thetransaction authorization request is based on user account informationand the transaction details. In an example, the merchant POS devicetransmits the transaction authorization request to the issuer system. Inthis example, the issuer system receives the transaction authorizationrequest, approves or denies the transaction authorization request, andtransmits either a denial of transaction authorization request or anapproval of the transaction authorization request to the accountmanagement system. For example, the issuer system identifies the userpayment account associated with the payment token. In an exampleembodiment, the transaction authorization request comprises a totaltransaction amount, and the issuer system determines whether thetransaction would result in the user exceeding the user's credit limitassociated with the user payment account. The issuer system may base adecision to approve a transaction authorization request based onconsiderations other than the total transaction amount or the user'scredit limit on the payment account.

In an example embodiment, the account management system transmitsnotification of an approved or denied transaction to the merchant POSdevice and/or to the user computing device. In this example embodiment,the merchant POS device and/or the user computing device displays orotherwise indicates to the user a status of the transaction. Forexample, the user computing device receives and displays a text messagefrom the account management system indicating to the user that thetransaction was denied.

In an example embodiment, the user associated with the user accountaudio template, and/or challenge and response for the user account, maywish to update user account data associated with the user account at atime after initially configuring a facial template. Updating accountinformation may comprise adding information associated with one or morenew payment accounts, editing information associated with one or moreexisting payment accounts, deleting data associated with one or moreexisting payment accounts, adding personal data, deleting personal data,and/or editing personal data. In this example, personal data maycomprise contact information of the user, demographic information of theuser, one or more preferences of the user, one or more permissionsgranting permission to the account management system to conduct one ormore actions with respect to hands free transactions, and/or otherrelevant personal data. In another example embodiment, updating accountinformation may comprise updating the facial template, audio template,or challenge and response associated with the user account. In anotherexample embodiment, the user computing device is stolen and anillegitimate user attempts to reconfigure the user account dataassociated with the user account so that the illegitimate user canengage in hands-free transactions using the user computing device of theuser.

In an example, the user requests to change user account data at amerchant POS device at the merchant location. In another exampleembodiment, the user requests to change user account data using the usercomputing device. In other example embodiments, user may request tochange user account data at another appropriate computing device. In theexample where the user requests to change user account data at themerchant POS device, the merchant POS device operator may select one ormore objects on the user interface of the merchant POS device to providean input indicating to the merchant POS device that the user desires tochange user account data. In this example, in response to receiving theinput of a request to change user account data associated with the useraccount, the merchant POS device transmits a request to the merchantcamera device to identify the user. In this example, the merchant cameradevice, in response to receiving the request to identify the user,captures a facial image of the user. In this example, the merchant POSdevice transmits the captured facial image to the account managementsystem via a network. In this example, the account management systemreceives the captured facial image via the network and generates afacial template based on the captured facial image. In this example, theaccount management system deletes the received captured facial imageafter successfully generating the facial template based on the capturedfacial image. In this example, the account management system comparesthe generated facial template against the existing facial templateassociated with the existing account.

An example method for updating user account data via the user computingdevice may be similar to the method for updating user account data viathe merchant POS device. For example, the user may request, by selectingone or more objects on a user interface of the user computing device, toprovide an input indicating to the user computing device that the userdesires to change user account data. In this example, in response toreceiving the input of a request to change user account data associatedwith the user account, the user computing device displays a request forthe user to capture a facial image of the user. In this example, theuser captures a facial image of the user via the user computing deviceby selecting an object on the user interface of the user computingdevice. In this example, the user computing device transmits thecaptured facial image to the account management system via a network. Inthis example, the account management system receives the captured facialimage via the network and generates a facial template based on thecaptured facial image. In this example, the account management systemdeletes the received captured facial image after successfully generatingthe facial template based on the captured facial image. In this example,the account management system compares the generated facial templateagainst the existing facial template associated with the existingaccount.

In an example embodiment, if the difference between the generated facialor audio template and the corresponding template previously associatedwith the user account is greater than or equal to a threshold amount,the account management system alerts the merchant POS device or usercomputing device that the facial template or audio template was notsuccessfully matched to the user account and that the user may notupdate user account information. In another example, if a similaritybetween the generated facial or audio template and the correspondingtemplate previously associated with the user account is less than athreshold amount, the account management system alerts the applicationon the merchant POS device or user computing device that the facialtemplate or audio template was not successfully matched to the useraccount and that the user may not update user account information. Ifthe difference between the generated facial or audio template and thecorresponding template previously associated with the user account isequal to or less than a threshold amount, the account management systemtransmits a notification to the POS device or the user computing devicethat the facial template or audio template was successfully matched tothe user account and the user may update user account information. Inanother example, if a similarity between the generated facial or audiotemplate and the corresponding template previously associated with theuser account is greater than or equal to a threshold amount, the accountmanagement system transmits a notification to the POS device or usercomputing device that the facial template or audio template wassuccessfully matched to the user account and the user may update useraccount information.

In response to receiving a notification from the account managementsystem that the facial template or audio template was successfullymatched to the user account and that the user may update accountinformation, the POS device or user computing device may display aninterface by which the merchant POS device operator or user may enterupdated account information. In this example, the user enters theupdated account information and the POS device or user computing devicetransmits the updated account information to the account managementsystem. Updating account information may comprise adding informationassociated with one or more new payment accounts, editing informationassociated with one or more existing payment accounts, deleting dataassociated with one or more existing payment accounts, adding personaldata, deleting personal data, and/or editing personal data. In thisexample, personal data may comprise contact information of the user,demographic information of the user, one or more preferences of theuser, one or more permissions granting permission to the accountmanagement system to conduct one or more actions with respect to handsfree transactions, and/or other relevant personal data. In anotherexample embodiment, updating account information may comprise updatingthe facial template, audio template, or challenge and responseassociated with the user account. If successfully updated, the user mayengage in hands-free transactions using the updated account informationassociated with the user account according to the example embodimentsdescribed herein. For example, in updating the user account information,the user added new payment account information. In this example, afterupdating the user account information, the user may conduct hands-freetransactions according to the methods described herein using the newpayment account information.

In certain example embodiments, when transmitting facial images to theaccount management system, the application on the merchant POS device orthe user computing device first sends a low resolution facial image tothe account management system. The account management system comparesthe low resolution facial image against the facial template associatedwith the account of the user. If the difference between the received lowresolution image and the corresponding facial template previouslyassociated with the user account is greater than or equal to a thresholdamount, the account management system alerts the application on themerchant POS device or user computing device that the facial templatedoes not match the facial template associated with the user account. Ifthe difference between the received low resolution image and thecorresponding facial template previously associated with the useraccount is greater than or equal to a threshold amount, the accountmanagement system transmits a request for a higher resolution image tothe application on the merchant POS device or user computing device. Theapplication on the merchant POS device or user computing device sends ahigh resolution facial image to the account management system. Theaccount management system generates a facial template and compares thegenerated template against the template associated with the account ofthe user. If the difference between the generated facial template andthe corresponding facial template previously associated with the useraccount is equal to or less than a threshold amount, the accountmanagement system transmits notification to the application on the usercomputing device or to the application on the merchant POS device thatthe user may update account data and the application or merchant POSdevice may display a request updated account data from the user, receivean input of updated account data from the user, and transmit the updatedaccount information to the account management system. The accountmanagement system may the associate the updated account information withthe user account. If successfully updated, the user may engage inhands-free transactions using the updated account data.

In other examples, in response to determining that the differencebetween the generated facial template and the corresponding facialtemplate currently associated with the user account is greater than athreshold amount or that a similarity between the generated facialtemplate and the corresponding facial template previously associatedwith the user account is less than a threshold amount, the accountmanagement system transmits, via the network, an alert to theapplication on to the merchant POS device or user computing device thatthe facial template was not successfully matched to the user account andrequests a response by the user to a challenge. In this example, theapplication on the merchant POS device or user computing device displaysthe challenge to the user and the user inputs the response. In thisexample, the application on the merchant POS device or user computingdevice transmits, to the account management system, the responsereceived from the user. The account management system verifies theresponse to the challenge. For example, the account management systemextracts the response associated with the challenge associated with theuser account and determines whether the response provided by the uservia the user computing device or merchant POS device matches, exactly orsubstantially, the response associated with the user account. Forexample, the responses match substantially if the account managementsystem determines that the similarity of response received from the userto the response associated with the user account is greater than orequal to a predefined threshold. In this example, the responses do notmatch substantially if the account management system determines that thesimilarity of the response received from the user to the responseassociated with the user account is less than a predefined threshold. Ifthe response provided by the user matches, exactly or substantially, theresponse associated with the user account, the account management systemassociates the generated user facial template or audio template with theuser account. If the response provided by the user does not match theresponse, exactly or substantially, associated with the user account,the account management system transmits an alert to the applicationnotifying the user that the facial template or audio template was notsuccessfully updated.

By using and relying on the methods and systems described herein, theaccount management system, the merchant beacon device, the usercomputing device, and the merchant POS device enable the user to conducta transaction with the merchant system without the user having tointeract with the user computing device or produce identity documents orphysical payment cards, as required in some current technology. As such,the systems and methods described herein may reduce the inputs requiredby the user via the user computing device and the inputs required by themerchant POS device operator to identify the user. Further, by using andrelying on the methods and systems described herein, the accountmanagement system and the user computing device may increase thesecurity of hands-free transactions by insuring that user biometricaccount data and payment account data, such as user images and userpayment account data, contact information, or demographic data, cannoteasily be changed by illegitimate users attempting to use the user'scomputing device in a hands free transaction.

Example System Architecture

Turning now to the drawings, in which like numerals indicate like (butnot necessarily identical) elements throughout the figures, exampleembodiments are described in detail.

FIG. 1 is a block diagram depicting a system 100 for processinghands-free transactions with facial recognition of a user 101 and forenabling the user 101 to update user 101 account information for use inhands-free transactions via a password comprising a facial template, inaccordance with certain example embodiments. As depicted in FIG. 1, thesystem 100 includes network computing devices 110, 120, 130, 150, and160 that are configured to communicate with one another via one or morenetworks 120. In some embodiments, a user associated with a device mustinstall an application and/or make a feature selection to obtain thebenefits of the techniques described herein.

In example embodiments, the network 140 can include a local area network(“LAN”), a wide area network (“WAN”), an intranet, an Internet, storagearea network (“SAN”), personal area network (“PAN”), a metropolitan areanetwork (“MAN”), a wireless local area network (“WLAN”), a virtualprivate network (“VPN”), a cellular or other mobile communicationnetwork, Bluetooth, Bluetooth low energy, NFC, or any combinationthereof or any other appropriate architecture or system that facilitatesthe communication of signals, data, and/or messages. Throughout thediscussion of example embodiments, it should be understood that theterms “data” and “information” are used interchangeably herein to referto text, images, audio, video, or any other form of information that canexist in a computer-based environment.

Each network computing device 110, 120, 130, 150, and 160 includes adevice having a communication component capable of transmitting andreceiving data over the network 140. For example, each network computingdevice 110, 120, 130, 150, and 160 can include a server, desktopcomputer, laptop computer, tablet computer, a television with one ormore processors embedded therein and/or coupled thereto, smart phone,handheld computer, personal digital assistant (“PDA”), or any otherwired or wireless, processor-driven device. In the example embodimentdepicted in FIG. 1, the network computing devices 110, 120, 130, 150,and 160 are operated by users 101, merchant beacon device 120 operators,merchant POS (“POS”) device 130 operators, issuer system 150 operators,and account management system 160, respectively.

An example user computing device 110 comprises an antenna 111, aBluetooth Low Energy (“BLE”) controller 112, a application 113, a userinterface 115, a data storage unit 116, a camera 117, a web browser 118,and a communication application 119.

In an example embodiment, the antenna 111 is a means of communicationbetween the user computing device 110 and a merchant beacon device 120.In an example embodiment, a BLE controller 112 outputs through theantenna 111 a radio signal, or listens for radio signals from themerchant beacon device 120. In another example embodiment a Bluetoothcontroller, Wi-Fi controller, or a near field communication (“NFC”)controller is used. In an example embodiment, the BLE controller 112outputs through the antenna 111 a radio signal, or listens for radiosignals from the merchant beacon device 120.

In an example embodiment, the BLE controller 112 is capable of sendingand receiving data, performing authentication and ciphering functions,and directing how the user computing device 110 will listen fortransmissions from the merchant beacon device 120 or configuring theuser computing device 110 into various power-save modes according toBLE-specified procedures. In another example embodiment, the usercomputing device 110 comprises a Bluetooth controller, Wi-Fi controlleror an NFC controller capable of performing similar functions. An exampleBLE controller 112 communicates with the application 113 and is capableof sending and receiving data over a wireless, BLE communicationchannel. In another example embodiment, a Bluetooth controller 112,Wi-Fi controller 112, or NFC controller 112 performs similar functionsas the BLE controller 112 using Bluetooth, Wi-Fi, or NFC protocols. Inan example embodiment, the BLE controller 112 activates the antenna 111to create a wireless communication channel between the user computingdevice 110 and the merchant beacon device 120. The user computing device110 communicates with the merchant beacon device 120 via the antenna111. In an example embodiment, when the user computing device 110 hasbeen activated, the BLE controller 112 polls through the antenna 111 aradio signal, or listens for radio signals from the merchant beacondevice 120.

In an example embodiment, the application 113 is a program, function,routine, applet, or similar entity that exists on and performs itsoperations on the user computing device 110. In certain exampleembodiments, the user 101 must install the application 113 and/or make afeature selection on the user computing device 110 to obtain thebenefits of the techniques described herein. In an example embodiment,the user 101 may access application 113 on the user computing device 110via the user interface 115. In an example embodiment, the application113 may be associated with the account management system 160. In anotherexample embodiment, the application 113 may be associated with amerchant system associated with the merchant beacon device 120 and/orthe merchant POS device 130.

In an example embodiment, the user interface 115 enables the user 101 tointeract with the application 113 and/or web browser 118. For example,the user interface 115 may be a touch screen, a voice-based interface,or any other interface that allows the user 101 to provide input andreceive output from an application, function, or component on the usercomputing device 110. In an example embodiment, the user 101 interactsvia the user interface 115 with the application 113 and/or web browser118 to configure user 101 accounts with the account management system160. In another example embodiment, the user 101 interacts via the userinterface 115 with the application 113 and/or the web browser 118 toenable hands-free payments, if needed.

In an example embodiment, the data storage unit 116 comprises a local orremote data storage structure accessible to the user computing device110 suitable for storing information. In an example embodiment, the datastorage unit 116 stores encrypted information, such as HTML5 localstorage.

In an example embodiment, the camera 117 may be any application,component, or function of the user computing device 110 that captures adigital image. The camera 117 may be resident on the user computingdevice 110 or in any manner logically connected to the user computingdevice 110. For example, the camera 117 may be connected to the usercomputing device 110 via the network 140. The camera 117 may be capableof obtaining individual images or a video scan. Any other suitable imagecapturing device may be represented by the camera 117.

In an example embodiment, the user 101 can use a communicationapplication 119, such as a web browser 118 application or a stand-aloneapplication, to view, download, upload, or otherwise access documents orweb pages via a distributed network 140.

In an example embodiment, the web browser 118 can enable the user 101 tointeract with web pages using the user computing device 110. In anexample embodiment, the user 101 may access the user's 101 accountmaintained by the account management system 160 via the web browser 118.In another example embodiment, the user 101 may access a merchant systemwebsite or an account management system website 169 via the web browser118. In certain example embodiments described herein, one or morefunctions performed by the application 113 may also be performed by aweb browser 118 application associated with the account managementsystem 160.

In an example embodiment, the communication application 119 can interactwith web servers or other computing devices connected to the network140, including a web server of a merchant system and a web server 168 ofthe account management system 160.

In certain example embodiments, one or more functions herein describedas performed by the application 113 may also be performed by a webbrowser 118 application or communication application 119, for example, aweb browser 118 application associated with a merchant system website orassociated with the account management system 160. In certain exampleembodiments, one or more functions herein described as performed by theapplication 113 may also be performed by the user computing device 110operating system. In certain example embodiments, one or more functionsherein described as performed via the web browser 118 may also beperformed via the application 113.

An example merchant beacon device 120 comprises an antenna 121 and aBluetooth Low Energy (“BLE”) controller 122. In an example embodiment, amerchant location comprises one or more merchant beacon devices 120installed at the merchant location. In an example embodiment, eachinstalled merchant beacon device 120 is associated by an accountmanagement system 160 with a particular merchant POS device 130installed at the merchant location. For example, the account managementsystem 160 may comprise a database that correlates merchant beacondevice 120 identifiers with merchant POS device 130 identifiers forassociated merchant POS devices 130. For example, a merchant POS device130 identifier may comprise hardware identifier specific to the devicesuch as a serial number or a media access control (“MAC”) identifier. Inanother example, a merchant beacon device 120 identifier may comprise ahardware identifier specific to the beacon device or an identifiergenerated by the account management system 160 and stored in themerchant beacon device 120. An example merchant beacon device 120 isprogrammed to broadcast, emit, or otherwise transmit a particularmerchant beacon device 120 identifier over a local wireless network, forexample, a BLE network, to any user computing devices 110 within athreshold distance required to maintain the wireless network 140. Forexample, the wireless network may comprise a BLE network 140, a Wi-Finetwork 140, a Bluetooth network 140, an NFC network 140, or any otherappropriate wireless network 140.

In an example embodiment, the antenna 121 is a means of communicationbetween the user computing device 110 and a merchant beacon device 120.In an example embodiment, a BLE controller 122 outputs through theantenna 121 a radio signal, or listens for radio signals from the usercomputing device 110. In another example embodiment a Bluetoothcontroller, Wi-Fi controller, or a near field communication (“NEC”)controller is used. In an example embodiment, the BLE controller 122outputs through the antenna 121 a radio signal, or listens for radiosignals from the user computing device 110.

In an example embodiment, the BLE controller 122 is capable of sendingand receiving data, performing authentication and ciphering functions,and directing how merchant beacon device 120 will listen fortransmissions from the user computing device 110 or configuring themerchant beacon device 120 into various power-save modes according toBLE-specified procedures. In another example embodiment, the merchantbeacon device 120 comprises a Bluetooth controller, Wi-Fi controller oran NFC controller capable of performing similar functions. An exampleBLE controller 122 communicates with the application 113 and is capableof sending and receiving data over a wireless, BLE communicationchannel. In another example embodiment, a Bluetooth controller 122, aWi-Fi controller 122, or an NFC controller 122 performs similarfunctions as the Wi-Fi controller 122 using Bluetooth, Wi-Fi, or NFCprotocols. In an example embodiment, the BLE controller 122 activatesthe antenna 121 to create a wireless communication channel between theuser computing device 110 and the merchant beacon device 120. Themerchant beacon device 120 communicates with the user computing device110 via the antenna 121. In an example embodiment, when the merchantbeacon device 120 has been activated, the BLE controller 122 pollsthrough the antenna 121 a radio signal, or listens for radio signalsfrom the user computing device 110.

An example merchant POS device 130 comprises an audio component 131, acamera 132, an application 133, a user interface 135, a data storageunit 136, and a communication application 139.

In an example embodiment, the audio component 131 may be any component,function, or application of the merchant POS device 130 that captures anaudio input of an external environment of the merchant POS device 130.The audio component 131 may be resident on the merchant POS device 130or in any manner logically connected to the merchant POS device 130. Forexample, the audio component 131 may be connected to the merchant POSdevice 130 via the network 140. The audio component 131 may be capableof obtaining an audio recording. Any suitable audio recording device maybe represented by the audio component 131. For example, the audiocomponent 131 may comprise a microphone.

In an example embodiment, the camera 132 may be any function, component,or application of the merchant POS device 130 that captures an image orvideo input of an external environment of the merchant POS device 130.The camera may be resident on the merchant POS device 130 or in anymanner logically connected to the merchant POS device 130. For example,the audio component 131 may be connected to the merchant POS device 130via the network 140. The camera 132 may be capable of capturing one ormore images or recording a video recording. Any suitable image capturingand/or video recording device may be represented by the camera 132.

In an example embodiment, the application 133 is a program, function,routine, applet, or similar entity that exists on and performs itsoperations on the merchant POS device 130. In certain exampleembodiments, the merchant POS (“POS”) device operator 102 or othermerchant system operator must install the application 133 and/or make afeature selection on the merchant POS device 130 to obtain the benefitsof the techniques described herein. In an example embodiment, themerchant POS device operator 102 may access the application 133 on themerchant POS device 130 via the user interface 135 of the merchant POSdevice 130. In an example embodiment, the application 133 may beassociated with the account management system 160. In another exampleembodiment, the application 133 may be associated with a merchant systemassociated with the merchant beacon device 120 and the merchant POSdevice 130.

In an example embodiment, the user interface 135 enables the merchantPOS device operator 102 to interact with the merchant POS device 130.For example, the user interface 135 may be a touch screen, a voice-basedinterface, or any other interface that allows the merchant POS deviceoperator 102 to provide input and receive output from an application,component, or function on the merchant POS device 130. In an exampleembodiment, the merchant POS device operator 102 interacts via the userinterface 135 with the application 133.

In an example embodiment, the data storage unit 136 comprises a local orremote data storage structure accessible to the merchant POS device 130suitable for storing information. In an example embodiment, the datastorage unit 136 stores encrypted information, such as HTML5 localstorage.

In an example embodiment, the communication application 139, such as aweb browser application or a stand-alone application, enables anoperator of the merchant POS device 130 to view, download, upload, orotherwise access documents or web pages via a distributed network 140.For example, the communication application 139 may enable communicationover the network 140 with the account management system 160, and/or anissuer system 150.

An example issuer system 150 approves or denies a payment authorizationrequest received from the merchant POS device 130. In an exampleembodiment, the issuer system 150 communicates with the merchant POSdevice 130 over the network 140. In an example embodiment, the issuersystem 150 communicates with an acquirer system to approve a creditauthorization for the user 101 and to make payment to the merchantsystem. For example, the acquirer system is a third party paymentprocessing system. In other example embodiments, the issuer system 150receives the payment authorization request from the account managementsystem 160 via the network 140.

An example account management system 160 comprises an account managementcomponent 161, a facial recognition component 163, an audio recognitioncomponent 165, a data storage unit 166, a transaction processingcomponent 167, a server 168, and a website 169.

In an example embodiment, the account management component 161 managesone or more user 101 accounts. In an example embodiment, a user 101account may comprise a digital wallet account, an email account, asocial networking account, or any other appropriate account associatedwith the account management system 160. In an example embodiment, theaccount management system 161 communicates with an application 113operating on a user computing device 110 associated with a user 101having a user 101 account maintained by the account management system160. In an example embodiment, the user 101 enters payment accountinformation into the user 101 account via the application 113. Theaccount management component 161 receives the payment accountinformation over the network 140 and associates the received paymentaccount information with the user 101 account.

In an example embodiment, the data storage unit 166 comprises a local orremote data storage structure accessible to the account managementsystem 160 suitable for storing information. In an example embodiment,the data storage unit 166 stores encrypted information, such as HTML5local storage.

In certain example embodiments, the transaction processing component 167receives transaction details from a merchant POS device 130 and arequest to initiate a transaction. Example transaction details comprisemerchant system account information, a total amount of the transaction,and a user 101 selection of a user 101 payment account associated withthe user's 101 account with the account management system 160. Forexample, the user's 101 account is a digital wallet account comprisinginformation for one or more payment accounts. In an example embodiment,the transaction processing component 167 extracts payment accountinformation from the user 101 account corresponding to the user 101selection of the user 101 payment account received in the transactiondetails from the merchant POS device 130. In an example embodiment, thetransaction processing component 167 transmits a payment authorizationrequest to an issuer system 150 or other appropriate financialinstitution associated with the payment account selected by the user 101for use in the transaction. An example payment authorization request maycomprise merchant system payment account information, user 101 paymentaccount information, and a total amount of the transaction. In anexample embodiment, after the issuer system 150 processes the paymentauthorization request, the transaction processing component 167 receivesan approval or denial of the payment authorization request from theissuer system 150 over the network 140. In an example embodiment, thetransaction processing component 167 transmits a receipt to the merchantPOS device 130 and/or the user computing device 110 comprising a summaryof the transaction.

It will be appreciated that the network connections shown are exampleand other means of establishing a communications link between thecomputers and devices can be used. Moreover, those having ordinary skillin the art having the benefit of the present disclosure will appreciatethat the user computing device 110, the merchant beacon device 120, themerchant POS device 130, the issuer system 150, and the accountmanagement system 160 illustrated in FIG. 1 can have any of severalother suitable computer system configurations. For example, a usercomputing device 110 embodied as a mobile phone or handheld computer mayor may not include all the components described above.

In example embodiments, the network computing devices and any othercomputing machines associated with the technology presented herein maybe any type of computing machine such as, but not limited to, thosediscussed in more detail with respect to FIG. 16. Furthermore, anyfunctions, applications, or modules associated with any of thesecomputing machines, such as those described herein or any others (forexample, scripts, web content, software, firmware, or hardware)associated with the technology presented herein may by any of themodules discussed in more detail with respect to FIG. 16. The computingmachines discussed herein may communicate with one another, as well aswith other computing machines or communication systems over one or morenetworks, such as network 170. The network 170 may include any type ofdata or communications network, including any of the network technologydiscussed with respect to FIG. 16.

Example Processes

The example methods illustrated in FIGS. 2-15 are described hereinafterwith respect to the components of the example operating environment 100.The example methods of FIGS. 2-15 may also be performed with othersystems and in other environments.

FIG. 2 is a block diagram depicting a method 200 for processinghands-free transactions with facial recognition of a user 101 and forenabling a user 101 to update user 101 account information for use inhands-free transactions via a password comprising a facial template, inaccordance with certain example embodiments. The method 200 is describedwith reference to the components illustrated in FIG. 1.

In block 210, the merchant system registers with the account managementsystem 160 and installs hardware in a merchant location. The method forregistering with an account management system 160 and installinghardware at a merchant location is described in more detail hereinafterwith reference to the method described in FIG. 3.

FIG. 3 is a block diagram depicting a method 210 for registering, by amerchant system, with an account management system 160 and installinghardware at a merchant location, in accordance with certain exampleembodiments. The method 210 is described with reference to thecomponents illustrated in FIG. 1.

In the example embodiments described herein, the merchant system doesnot need to install hardware at the example merchant location in anyparticular order. The method 210 describes one example method ofinstalling hardware at the merchant location. However, the merchantsystem or other system installing the merchant hardware does not need toinstall the merchant POS device 130, the merchant POS device 130, or themerchant beacon device 120 in the order described herein.

In block 310, a merchant system registers with the account managementsystem 160. In an example embodiment, an agent of the merchant systemaccesses an account management system 160 website and registers for amerchant account with the account management system 160 via the website.In an example embodiment, the merchant system adds payment accountinformation associated with a merchant account to the merchant accountmanaged by the account management system 160. In an example embodiment,the merchant system comprises one or more merchant locations. Forexample, the merchant system may comprise one or more physical storelocations. An example merchant location comprises one or more merchantPOS (“POS”) devices 130. In an example embodiment, one or more merchantPOS device operators 102 operate the one or more merchant POS devices130 at the merchant location.

In block 320, a merchant system operator installs the application 133 onthe merchant POS device 130. In another example embodiment, the merchantsystem operator purchases a merchant POS device 130 from the accountmanagement system 160 with the application 133 pre-installed on themerchant POS device 130. In an example embodiment, the merchant POSdevice 130 is able to communicate with the account management system 160over a network 140. In an example embodiment, the merchant POS device130 communicates with the account management system 160 via theapplication 133. For example, the merchant POS device 130 may be able totransmit transaction details to the account management system 160 viathe application 133 over the network 140 to enable the accountmanagement system 160 to process a transaction. In another example, themerchant POS device 130 receives a receipt from the account managementsystem 160 that notifies a merchant POS device operator 102 as towhether a transaction was successful or not.

In block 330, the merchant beacon device 120 receives a beaconidentifier from the account management system 160. In an exampleembodiment, the merchant system receives a beacon identifier from theaccount management system 160 and installs or otherwise saves the beaconidentifier on the merchant beacon device 120. In an example embodiment,a merchant system operator installs the merchant beacon device 120 inproximity to a merchant POS device 130. In an example embodiment, themerchant system operator installs a plurality of merchant beacon devices120, each merchant beacon device 120 in proximity to one or moreassociated merchant POS devices 130. In an example embodiment, themerchant beacon device 120 is able to broadcast a merchant beaconidentifier over a wireless medium. One or more user computing devices110 located within a threshold proximity to the merchant beacon device120 are able to receive the merchant beacon identifier over the wirelessmedium. In another example embodiment, the merchant beacon device 120 isable to establish a local network 140 connection to one or more usercomputing devices 110 located within a threshold proximity to themerchant beacon device 120 and the merchant beacon device 120 transmitsthe merchant beacon identifier to the one or more user computing devices110 over the established local network 140 connection. For example, thethreshold proximity depends on the network 140 communication protocolutilized by the merchant beacon device 120.

In block 340, the merchant beacon device 120 broadcasts the beaconidentifier code via a wireless communication at the merchant location.For example, the merchant beacon device 120 may broadcast, emit, orotherwise transmit data comprising the beacon identifier via Wi-Fi,Bluetooth, Bluetooth low energy (“BLE”), near field communication(“NFC”), or other appropriate communication protocol to one or more usercomputing devices 110 located within a threshold proximity to themerchant beacon device 120 at the merchant location. In some exampleembodiments, before transmitting the merchant beacon identifier, themerchant beacon device 120 is operable to establish a network 140connection between the merchant beacon device 120 and one or more usercomputing devices 110 located within a threshold proximity to themerchant beacon device 120 at the merchant location.

In block 350, a merchant system operator installs the merchant POSdevice 130 at the merchant location to correspond to the merchant beacondevice 120. In an example embodiment, both a merchant camera device anda merchant beacon device 120 are installed in proximity to a particularmerchant POS device 130. In another example embodiment, a merchant POSdevice 130 and a merchant beacon device 120 are installed in proximityto two or more particular merchant POS devices 130. In an exampleembodiment, the merchant POS device 130 is oriented to be able tocapture video and/or images of a face of a user 101 standing in front ofone or more merchant POS devices 130 during the process of checkout. Inan example embodiment, the merchant system installs a merchant POSdevice 130 that is oriented to capture video and/or images of the faceof a user standing in front of a particular merchant POS device 130. Inanother example embodiment, the merchant system installs a merchant POSdevice 130 that is oriented to capture video and/or images of the facesof one or more users 101 standing within a proximity of a particularplurality of merchant POS devices 130 within a range of a field ofvision of the camera 147 of the merchant POS device 130.

In block 360, the account management system 160 receives a merchant POSdevice 130 identifier and associates it with the corresponding beaconidentifier code of the merchant beacon device 120. In an exampleembodiment, the merchant system and/or the account management system 160configures the merchant POS device 130 so that the merchant POS device130 is able to communicate with the account management system 160 overthe network 140. An example camera device 140 identifier comprises ahardware identifier, a MAC address, or other useful or relevantidentifier associated with the merchant POS device 130. In an exampleembodiment, the account management system 160 comprises a database thatlists merchant POS device 130 identifiers and associated beaconidentifiers for merchant beacon device 120 identifiers for a particularmerchant location. In an example embodiment, the merchant POS device 130transmits the merchant beacon device 120 identifier and the merchant POSdevice 130 identifier to the account management system 160. In anexample embodiment, during the setup and installation process, themerchant POS device 130 receives the merchant beacon device 120identifier over an appropriate wireless communication channel from themerchant beacon device 120. In another example embodiment, during thesetup and installation process, the merchant POS device 130 establishesa network 140 connection with the merchant beacon device 120 andreceives the merchant beacon device 120 identifier over the network 140.In another example embodiment, the account management system 160receives the merchant POS device 130 identifier, extracts one or moremerchant beacon device 120 identifiers from the database, and associatesthe merchant POS device 130 identifier with one or more of the one ormore extracted merchant beacon device 120 identifiers. In yet anotherexample embodiment, the merchant system operator installs the one ormore merchant beacon devices 120 after installing the one or moremerchant camera devices 140. In this example embodiment, the accountmanagement system 160 generates a merchant beacon device identifier toassociate with a merchant POS device 130 identifier and transmits thegenerated merchant beacon device 120 identifier to the merchant system.In this example embodiment, the merchant system operator manuallyconfigures the merchant beacon device 120 to broadcast, emit, orotherwise transmit the merchant beacon device identifier assigned by theaccount management system 160 over a network 140.

In certain example embodiments, one or both of the merchant POS device130 and the merchant beacon device 120 are components of the merchantPOS device 130 or are wirelessly or physically connected to the merchantPOS device 130 and controlled by one or more processors of the merchantPOS device 130. In certain example embodiments, certain functionsdescribed herein as performed by the merchant POS device 130 and/or themerchant beacon device 120 may also be performed by the merchant POSdevice 130.

From block 360, the method 210 proceeds to block 220 of FIG. 2.

Returning to FIG. 2, in block 220, the user 101 registers with theaccount management system 160. The method for registering for an accountwith an account management system 160 is described in more detailhereinafter with reference to the method 220 described in FIG. 4.

FIG. 4 is a block diagram depicting a method 220 for registering, by auser 101, for an account with an account management system 160, inaccordance with certain example embodiments. The method 220 is describedwith reference to the components illustrated in FIG. 1.

In block 410, the user 101 accesses the account management systemwebsite 169. For example, the user 101 accesses the account managementsystem 160 via the web browser 118 of the user computing device 110. Inanother example, the user 101 may otherwise contact the accountmanagement system 160 to register for a user 101 account.

In block 420, the user 101 registers with the account management system160. The user 101 may obtain a user account number, receive theappropriate applications and software to install on the user computingdevice 110, request authorization to participate in hands-free paymentprocessing, or perform any action requested by the account managementsystem 160. The user 101 may utilize the functions of the user computingdevice 110, such as the user interface 115 and the web browser 118, toregister and configure a user 101 account. In an example embodiment, theuser 101 enters user account information into the user account via theuser computing device 110. Example user account information comprisespayment account information associated with one or more user 101accounts, for example, one or more credit accounts, one or more bankaccounts, one or more stored value accounts, and/or other appropriateaccounts. Example user account information may also comprise user 101contact information such as a user 101 residential address, commercialaddress, business address, telephone number, email address, or otherappropriate contact information. Example user 101 account informationmay also comprise demographic information of the user 101 such as an ageof the user, membership status of the user 101 in rewards programs orclubs, or other relevant demographic information of the user 101.Further, example user account information comprises a user facialtemplate or audio template as described in the example embodimentsherein. Further, example user 101 account information comprises anoption to enable or disable a permission of the account managementsystem 160 to process hands free transactions as described in theexample embodiments herein.

In block 430, the user 101 downloads the application 113 onto the usercomputing device 110. In an example embodiment, the application 113operating on the user computing device 110 is able to communicate withthe account management system 160 over the network 140. In an exampleembodiment, the user 101 configures the user 101 account settings oradd, delete, or edit user account information via the application 113.In an example embodiment, the user 101 selects an option to enable ordisable the permission of the account management system 160 to processhands free transactions. For example, a hands free transaction comprisesa transaction wherein the user 101 does not need to interact with theuser computing device 110 or that requires minimal user 101 interactionwith the user computing device 110 to initiate a transaction with themerchant system.

In block 440, the account management system 160 establishes a facialtemplate associated with the user 101 account. The method forestablishing a facial template associated with a user 101 account isdescribed in more detail hereinafter with reference to the method 440described in FIG. 5.

FIG. 5 is a block diagram depicting a method 440 for establishing afacial template associated with a user 101 account, in accordance withcertain example embodiments. The method 440 is described with referenceto the components illustrated in FIG. 1.

In block 510, the application 113 displays a request for the user 101 tocapture a facial image via the user computing device 110. In an exampleembodiment, the application 113 displays the request via the userinterface 115. In an example, the user interface 115 may display arequest that reads, “to enable hands free transactions, we need an imageof your face. Would you like submit a facial image now?” In thisexample, the user 101 selects an option to take a current picture orotherwise selects a picture stored on the user computing device 110.

In block 520, the user 101 selects an option to capture a facial image.For example, the user 101 actuates an object on the user interface 115that reads, “yes, I would like to take a picture now.”

In block 530, the application 113 activates the camera 117 on the usercomputing device 110 and the user 101 captures a facial image of himselfIn an example embodiment, the user computing device user interface 115displays a live camera feed of the user 101 to aid the user 101 inaligning the user's 101 face to take the facial image. In an exampleembodiment, the application 113 displays a box or other perimeter on theuser interface 115 within which the user 101 should align his face totake a picture of a required size predetermined by the accountmanagement system 160. In an example embodiment, the user 101 actuatesan object on the user interface 115 to capture the image. In thisexample embodiment, in response to the user actuating the object on theuser interface 115, the camera 117 receives a command from theapplication 113 to capture an image of the user 101. In another exampleembodiment, the camera 117 receives a command from the application 113to capture a plurality of images of the user 101 as the user 101 movesthe camera around the user's 101 face. For example, each of theplurality of images of the user 101 corresponds to a particular pose ofthe user's 101 face. An example facial image may comprises a digitalimage of the face of a user 101. In an example embodiment, the accountmanagement system 160 establishes guidelines for users 101 in submittingfacial images. For example, the application 113 directs the user 101 toremove any hats, head coverings, glasses, or other objects oraccessories that may occlude regions of the user's 101 face so thatapplication 160 may receive a complete depiction of the user's 101 face.

In an example embodiment, the user computing device 110 determines ifthe captured facial image is a valid facial image or an invalid facialimage. For example, a valid facial image complies with guidelinespredetermined by the account management system 160 and an invalid facialimage does not comply with one or more of the guidelines. For example,if the user computing device 110 captures a facial image that comprisesincorrect dimensions, if part or all of the user's 101 face is occluded,or if the image is too dark or too bright, the user computing device 110rejects the invalid facial image and displays a request directing theuser 101 to capture a subsequent facial image. In this example, the user101 captures a subsequent facial image via the user computing device110, and the user computing device 110 transmits the subsequent facialimage to the account management system 160 via the network 140.

In block 540, the account management system 160 receives the facialimage. In another example embodiment, the account management system 160receives a plurality of facial images of the user 101. For example, theapplication 113 transmits the one or more facial images of the user 101to the account management system 160 via the network 140. In an exampleembodiment, the account management system 160 associates the receivedone or more facial images with the user 101 account. For example, theaccount management system 160 is able to identify the user 101 accountto associate with the received one or more images because the user 101is currently logged in to the application 113 on the user computingdevice 110 at the time the one or more facial images are transmitted tothe account management system 160. In certain example embodiments, theaccount management system 160 determines if the received facial image isa valid facial image or an invalid facial image. For example, a validfacial image complies with all guidelines predetermined by the accountmanagement system 160 and an invalid facial image does not comply withone or more of the guidelines. For example, if a user 101 submits afacial image that comprises incorrect dimensions, if part or all of theuser's 101 face is occluded, or if the image is too dark or too bright,the account management system 160 rejects the invalid facial image andtransmits a request to the user computing device 110 directing the user101 to capture a subsequent facial image to transmit to the accountmanagement system 160. In this example, the user computing device 110receives and displays the request. The user 101 captures a subsequentfacial image via the user computing device 110 and transmits thesubsequent facial image to the account management system 160 via thenetwork 140. In another example, the user 101 submits a facial imagethat is not of a face. The account management system 160 or application113 determines, via facial recognition, that the image is not of a face,the account management system 160 or application 113 rejects the invalidfacial image, and transmits a request to the user computing device 110for display by the user computing device 110 directing the user 101capture a subsequent facial image to transmit to the account managementsystem 160. In this example, the user computing device 110 receives anddisplays the request, the user 101 captures a subsequent facial imagevia the user computing device 110, and the user computing device 110transmits the subsequent facial image to the account management system160 via the network 140. In yet another example, the user 101 submits afacial image that is of a face, but the account management system 160 orapplication 113 determines that the image, based on one or more imagemetrics such as image resolution, is not of a minimum quality standard.The account management system 160 or application 113 rejects the invalidfacial image and transmits a request to the user computing device 110for display by the user computing device 110 directing the user 101capture a subsequent facial image to transmit to the account managementsystem 160. In this example, the user computing device 110 receives anddisplays the request. The user 101 captures a subsequent facial imagevia the user computing device 110, and the user computing device 110transmits the subsequent facial image to the account management system160 via the network 140.

In block 550, the account management system 160 creates a facialtemplate associated with the user 101 account based on the receivedfacial image. In another example embodiment, the account managementsystem 160 generates a corresponding facial template for each of aplurality of received facial images associated with the user 101account. In an example, the facial template is of a predetermined size,for example, a 128-byte facial template. In an example, the accountmanagement system 160 generates a facial template comprising a computercode representation of the digital facial image. For example, the facialtemplate may describe key features of the facial image of the user 101,such as shape, color, line, value, space, form, texture, or other usefulor relevant feature of the image or of particular regions of the image.In an example, the facial template is generated by processing the facialimage through a convolutional neural network.

In an example embodiment, the account management system 160 stores thegenerated facial template associated with the user 101 in a data storageunit 166 associated with the account management system 160. For example,the account management system 160 database comprises a table or othermeans by which it correlates each user 101 account identifier with anassociated facial template of the user 101.

In another example embodiment, after the user computing device 110captures one or more facial images of the user 101, the user computingdevice 110 generates one or more facial templates corresponding to oneor more of the one or more captured facial images of the user 101. Inthis example embodiment, the user computing device 110 transmits the oneor more generated facial templates to the account management system 160over the network 140.

In block 560, the account management system 160 deletes the receivedfacial image. For example, the account management system 160 only uses afacial template comprising a computer code representation of the facialimage of the user 101. In another example, the account management system160 saves the received facial image for future processing. For example,the account management system 160, at a later time, updates a facialtemplate generation algorithm and generates an updated facial templatecorresponding to the saved facial image.

From block 560, the method 440 proceeds to block 450 in FIG. 4.

Returning to block 450, in FIG. 4, the account management system 160establishes an audio template associated with the user 101 account. Themethod for establishing an audio template associated with a user 101account is described in more detail hereinafter with reference to themethod 450 described in FIG. 6.

FIG. 6 is a block diagram depicting a method 450 for establishing anaudio template associated with a user 101 account, in accordance withcertain example embodiments. The method 450 is described with referenceto the components illustrated in FIG. 1.

In block 610, the application 113 displays a request for the user 101 tocapture an audio recording of the user's 101 voice via the usercomputing device 110. In an example embodiment, the application 113displays the request via the user interface 115. In an example, the userinterface 115 may display a request that reads, “to enable hands freetransactions, we need recording of your voice. Would you like submit anaudio recording now?” In this example, the user 101 selects an option tosubmit a live audio recording or otherwise selects a pre-recorded audiorecording of the user 101 stored on the user computing device 110.

In block 620, the user 101 selects an option to capture an audiorecording. For example, the user 101 actuates an object on the userinterface 115 that reads, “yes, I would like to submit a audio recordingnow.”

In block 630, the application 113 activates an audio module (notdepicted) on the user computing device 110 and the user 101 captures anaudio of the user's 101 voice. In an example embodiment, the usercomputing device user interface 115 may display guidelines received fromthe account management system 160 to aid the user 101 in submitting anaudio recording. For example, the application 113 may display directionsfor the user 101 to record the user's 101 voice saying the user's 101name. In another example, the application 113 directs the user 101 tocapture an audio recording to establish a voice password that can beused in hands-free transactions of the user 101. In an exampleembodiment, the user 101 may actuate an object on the user interface 115to capture or record a live audio recording. In this example embodiment,in response to the user actuating the object on the user interface 115,the audio module receives a command from the application 113 to capturean audio recording of the user 101. In an example embodiment, theaccount management system 160 establishes guidelines for users 101 insubmitting audio recordings. For example, the application 113 directsthe user 101 to submit an audio recording no longer than a predefinedmaximum length of duration. For example, the audio recording must be nolonger than five seconds. In another example, the application 113directs the user 101 submit an audio recording lasting a predefinedlength of time, for example, an audio recording lasting five seconds. Inan example embodiment, after the user 101 initiates an audio recordingby actuating an object of the user interface 115, the application 113transmits a command to the audio module to stop recording the user's 101voice after the predefined length of time or predefined maximum lengthof time has expired.

In block 640, the account management system 160 receives an audiorecording of the voice of the user 101. For example, the application 113or audio module transmits the audio recording to the account managementsystem 160 over the network 140. In another example, the application 113retrieves an audio recording selected by the user 101 and saved in thedata storage unit 116 and transmits the retrieved audio recording to theaccount management system 160 over the network 140.

In block 650, the account management system 160 creates an audiotemplate associated with the user's 101 voice based on the receivedaudio of the voice of the user. In an example, the audio template is ofa predetermined size. In an example, the account management system 160generates an audio template comprising a computer code representation ofthe user's 101 audio recording. For example, the audio template maydescribe key features of the audio recording of the user 101, such asthe intonation of the user's 101 voice or other features of the user's101 voice. Example features of a user's voice comprise a volume, apitch, and/or other features of the user's 101 voice.

In an example embodiment, the account management system 160 stores thegenerated audio template associated with the user 101 in a data storageunit 166 associated with the account management system 160. For example,the account management system 160 database comprises a table or othermeans by which it correlates each user 101 account identifier with anassociated audio template of the user 101.

In block 660, the account management system 160 deletes the receivedaudio of the voice of the user 101. In an example embodiment, theaccount management system 160 deletes the received audio recording ofthe user 101 to protect the privacy of the user 101. For example, theaccount management system 160 only uses an audio template comprising acomputer code representation of the audio recording of the user 101.

From block 660, the method 440 proceeds to block 230 in FIG. 2.

Returning to block 230, in FIG. 2, the user device receives a merchantbeacon device 120 identifier. The method for receiving a merchant beaconidentifier broadcast by a merchant beacon device 120 is described inmore detail hereinafter with reference to the method 240 described inFIG. 7.

FIG. 7 is a block diagram depicting a method 240 for receiving, by auser computing device 110, a merchant beacon identifier broadcast by amerchant beacon device 120, in accordance with certain exampleembodiments. The method 240 is described with reference to thecomponents illustrated in FIG. 1.

In block 710, the user 101 enters the merchant location and signs intothe application 113 on the user computing device 110. In another exampleembodiment, the user 101 signs into the application 113 at a time beforeentering the merchant location and enters the merchant location carryingthe user computing device 110 signed into the application 113.

In block 720, the user 101 signs in to the application 113 on the usercomputing device 110. In an example embodiment, the user 101 has ausername and password associated with the user 101 account maintained bythe account management system 160. In an example embodiment, the user101 opens the application 113 on the user computing device 110 andenters a username and/or password via the user interface 115 to sign into the application 113. In an example embodiment, when the user 101 issigned in to the application 113, the application is able to communicatewith the account management system 160 over the network 140. In thisexample embodiment, when the user 101 is not signed in to theapplication 113, the application does not communicate with the accountmanagement system 160 even if the a network 140 connection is available.In an example embodiment, the user 101 may sign out of the application113 at any time by actuating one or more objects on the user interface115 of the user computing device 110. In an example embodiment, aftersigning in to the application 113, the user 101 configures one or moreuser 101 account settings, add, edit, or delete user 101 payment accountinformation, and/or change user 101 preferences. In certain exampleembodiments, a user 101 may be required to make a feature selection toobtain the benefits of the techniques described herein. For example, theuser 101 may have to enable one or more user 101 account settings toenable hands free transactions according to the methods describedherein.

In an example embodiment, application 113 provides options, data,configurable alerts, and other suitable features to the user 101. Forexample, the application 113 comprises a listing of merchant systems andmerchant locations that participate in hands free payment transactionsaccording to one or more of the methods described herein. The listingmay be updated periodically from the account management system 160. Theapplication 113 may notify the user 101 when the user 101 is within aconfigured vicinity of a participating merchant system. The application113 may provide the user 101 with options to update payment preferences.The application 113 may provide the user 101 with a listing of recenttransactions. The application 113 may provide any other suitableinformation to the user 101.

In block 730, the user 101 carries the user computing device 110 withina threshold distance of the merchant beacon device 120 at the merchantlocation. In an example embodiment, the user 101 enters the merchantlocation carrying the user computing device 110 in a pocket or a bag, inthe hands of the user 101, or in any suitable manner. The location ofthe merchant system may be a store location, a kiosk location, or anysuitable physical location of a merchant system. In another exampleembodiment, a merchant POS operator 102 may be mobile and arrive at alocation of the user 101. For example, the merchant system may be arestaurant and the merchant POS device operator 102 may be a deliveryperson possessing a portable merchant POS device 130.

In certain example embodiments, the application 113 alerts the user 101when the user 101 is in the vicinity of a merchant location that acceptshands-free payments. The alert may be provided via a message on the usercomputing device 110, via an email or a text, or in any suitable manner.In an example embodiment, the alert is based on the location of the user101 as determined by a GPS component (not depicted) resident on the usercomputing device 110. For example, the application 113 accesses the GPSdata from the GPS component and compare the GPS location to a list ofmerchant locations that accept hands free payments. For example, theapplication 113 comprises a list or accesses a list maintained by theaccount management system 160 of merchant locations that accept handsfree payments. If a match results from the comparison, then an alert isgenerated and provided to the user 101. The match may result if the user101 is within a configured distance of a qualified merchant location. Inan example embodiment, the alerts may be configured to alert in anysuitable manner. In an example, the alerts may be combined incommercially dense environments or the alerts may be presentedindividually. In another example, the alerts may be configured to onlyalert the user 101 a configured number of times. For example, the alertmay be presented three times, but upon a fourth instance, the alert isnot presented. The alerts may be presented as a notification with anaudible alert, a vibration, a popup alert on the user interface 115 ofthe user computing device 110, or other suitable alert.

In block 740, the user computing device 110 receives a merchant beaconidentifier broadcasted by the merchant beacon device 120. The usercomputing device 110 recognizes a merchant beacon device 120 viawireless communication at the location of the merchant system. The usercomputing device 110 may be configured to search for beacons or otherwireless signals. In an example embodiment, the user computing device110 and the merchant beacon device 120 establish a BLE wireless network140 connection. In other example embodiments, the user computing device110 and the merchant beacon device 120 establish a Bluetooth, Wi-Fi,NFC, or other appropriate network 140 connection. Upon entering therange of the signal of the merchant beacon device 120, the usercomputing device 110 receives the merchant beacon identifier.

In block 750, the user computing device 110 transmits the receivedmerchant beacon identifier and a user 101 account identifier to theaccount management system 160. In an example embodiment, the usercomputing device 110 transmits the data received in the merchant beaconidentifier along with a user 101 account identifier to the accountmanagement system 160 over the network 140.

In block 760, the account management system 160 receives the merchantbeacon identifier and the user 101 account identifier. For example, theaccount management system 160 receives the merchant beacon identifierand the user 101 account identifier over the network 140. The usercomputing device 110 may compare data from the merchant beaconidentifier to a database of merchant beacon identifier data and merchantcamera device identifier data. In an example embodiment, the usercomputing device 110 compares this data to determine an identity of themerchant system and an identity of the merchant POS device 130associated with the merchant beacon identifier. In another exampleembodiment, the user computing device 110 compares this data to verifythe authenticity of the beacon.

From block 760, the method 230 proceeds to block 240 in FIG. 2.

Returning to FIG. 2, in block 240, the merchant POS device 130 receivesa facial template for each user 101 in range of the merchant beacondevice 120. The method for receiving, by a merchant POS device 130, afacial template for each user 101 in range of the merchant beacon device120 is described in more detail hereinafter with reference to the method240 described in FIG. 8. In other example embodiments, in addition to orinstead receiving the facial template, the merchant POS device 130receives an audio template and/or a challenge and response associatedwith the user 101 account.

FIG. 8 is a block diagram depicting a method 240 for receiving, by amerchant POS device 130, a facial template for each user 101 in range ofthe merchant beacon device 120, in accordance with certain exampleembodiments. The method 240 is described with reference to thecomponents illustrated in FIG. 1. In other example embodiments, inaddition to or instead receiving the facial template, the merchant POSdevice 130 receives an audio template and/or a challenge and responseassociated with the user 101 account according to a similar method.

In block 810, the account management system 160 extracts a facialtemplate, audio template, and/or challenge and response associated withthe user 101 account identifier. In an example embodiment, the accountmanagement system 160 accesses a database comprising stored facialtemplates for a plurality of users 101 with corresponding user 101account identifiers for each user 101. For example, this database isstored in the data storage unit 166. In another example, the accountmanagement system 160 extracts an audio template and/or a challenge andresponse instead of or in addition to the facial template from thedatabase.

In block 820, the account management system 160 generates a paymenttoken for a user payment account and notifies an issuer system thatmaintains the payment account of the payment token associated with thepayment account. In an example embodiment, the account management system160 generates a payment token for each user 101 whose user computingdevice 110 is within network range of the merchant beacon device 120 andis signed in to the application 113. An example payment token comprisesa series of alphanumeric and/or symbolic characters. The example paymenttoken may be associated with a payment account of the user 101 and berecognizable by an issuer system 150 associated with the payment accountof the user 101. For example, the account management system 160generates the payment token and communicates the payment token to theissuer system 150 associated with the payment account. In this example,if the issuer system 150 receives the payment token from a POS device130 in a payment transaction, the issuer system 150 is able to extractthe user 101 payment account information associated with the paymenttoken.

In some example embodiments, the account management system 160 placesrestrictions on payment tokens for security reasons or according to oneor more configurations of the user 101 account desired by the user 101.For example, the payment token may only be valid for a preconfiguredlength of time, for example, one hour. In another example, the paymenttoken may only be valid for us in a transaction between the user 101 anda particular merchant system. In yet another example, the payment tokenis only valid for use within a particular geographic boundary or withina threshold distance from a geographic point. In an example embodiment,the account management system 160 communicates one or more of theseexample restrictions to the issuer system 150 and the issuer system 150associates these one or more restrictions with the payment token and theuser 101 payment account data in a database of the issuer system 150. Inan example embodiment, the account management system 160 communicates atime stamp representing a time when the payment token was generated tothe issuer system 150. In another example embodiment, the accountmanagement system 160 communicates location data describing geographicboundaries and/or threshold distances from geographic points where thepayment token may be used in a transaction to the issuer system 150. Inyet another example embodiment, the account management system 160communicates a merchant system identifier and instructions that onlypayment authorization requests originating from merchant systemscomprising the merchant system identifier may be approved to the issuersystem 150. In an example embodiment, the issuer system 150 associatesthe payment token, the user 101 payment account data associated with thepayment token, the one or more restrictions placed on the payment tokenby the account management system 160, and/or one or more of locationdata, time stamp data, merchant system identifier data, or other datathat the issuer system 150 may use to determine whether the one or morerestrictions on the payment token are satisfied to enable use of thepayment token.

In block 830, the account management system 160 identifies a merchantPOS device 130 associated with the merchant beacon device 120identifier. In an example embodiment, the account management system 160recognizes that the merchant beacon identifier is associated with theaccount management system 160 and a particular merchant POS device 130at the merchant location. In an example embodiment, the accountmanagement system 160 recognizes that the merchant beacon identifier isassociated with a plurality of merchant POS device 130 installed at amerchant location.

In block 840, the account management system 160 transmits the facialtemplate of the identified user 101, the audio template of theidentified user 101, and/or the challenge and response associated withthe identified user 101, along with the generated payment token to themerchant POS device 130 associated with the merchant beacon device 120identifier. In another example embodiment, the account management system160 transmits the facial template of the identified user 101 and thegenerated payment token to a plurality of merchant POS devices 130associated with the merchant beacon device 120 identifier. In certainexample embodiments, the account management system 160 receives, in realtime or near real time, a plurality of transmissions from a plurality ofuser computing devices 101 corresponding to a plurality of users 101present at the merchant location. Each transmission comprises a user 101account identifier and a retransmitted merchant beacon identifier. Inthese example embodiments, in response to receiving each suchtransmission, the account management system 160 retrieves a facialtemplate associated with the received user 101 account identifier andtransmits a facial template to one or more merchant POS devices 130 atthe merchant location associated with the merchant beacon identifier. Inother example embodiments, in addition to or instead transmitting facialtemplate, the account management system 160 transmits an audio templateand/or a challenge and response associated with the user 101 account toone or more merchant POS devices 130.

In block 850, the merchant POS device 130 receives the facial templateof the user 101. In another example, in addition to or instead receivingthe facial template, the merchant POS device 130 receives an audiotemplate and/or a challenge and response associated with the user 101account. In another example embodiment, a plurality of merchant POSdevices 130 receives the facial template of the user 101. In yet anotherexample embodiment, the merchant POS devices and/or the plurality ofmerchant POS devices 130 receives one or more additional facialtemplates, audio templates, and/or challenges and responses from theaccount management system 160 corresponding to one or more users 101other than the instant user 101 having user computing devices 110 innetwork 140 connection to a merchant beacon device 120 according to themethod previously described herein. For example, the one or moreadditional facial templates, audio templates, and/or challenges andassociated responses are received in real time from the accountmanagement system 160 as additional users 101 other than the instantuser 101 receive the merchant beacon device 120 identifier over awireless communication network 140 or otherwise establish a network 140connection between their user computing devices 110 and one or moremerchant beacon devices 120. For example, the one or more merchant POSdevices 130 may receive one or more additional facial templates, audiotemplates, and/or challenges and responses corresponding to one or moreadditional users 101 at a time before, at the same time, or after thetime at which the merchant POS devices 130 receives the facial templateof the instant user 101.

In block 860, the merchant POS device 130 adds the facial template ofthe user 101 to a customer log. In an example, the merchant POS device130 adds, in addition to or instead of the facial template of the user101, an audio template and/or a challenge and response associated withthe user 101 to the customer log. In an example embodiment, the customerlog is accessible by the merchant POS device 130 and by the accountmanagement system 160. In an example embodiment, the merchant POS device130 maintains the customer log on the merchant POS device 130 or on acomputing device logically connected to the merchant POS device 130. Inanother example embodiment, the account management system 160 adds thefacial template of the user 101 to the customer log. In this exampleembodiment, the account management system 160 maintains the customer login a database or storage device of the account management system 160.

In block 870, the merchant POS device 130 periodically updates thecustomer log. The method for updating a customer log as users 101 enteror leave a network range of a merchant beacon device 120 is described inmore detail hereinafter with reference to the method 870 described inFIG. 9.

FIG. 9 is a block diagram depicting a method 870 for receiving, by amerchant POS device 130, notification from an account management system160 as users 101 enter or leave a network range of a merchant beacondevice 120, in accordance with certain example embodiments. The method870 is described with reference to the components illustrated in FIG. 1.

In block 910, the merchant POS device 130 is notified by the accountmanagement system 160 as users 101 signed into a payment account enteror leave a network range of the merchant beacon device 120. For example,as previously discussed, when a user 101 carrying a user computingdevice 110 enters a threshold distance from a merchant beacon device120, the merchant beacon device 120 or the user computing device 110 ofthe user 101 are able to detect the other device and establish awireless network 140 connection between the two devices at the merchantlocation. In this example, the merchant beacon device 120 transmits themerchant beacon identifier corresponding to the merchant beacon device120 over the local wireless network 140 to the user computing device110. For example, the merchant beacon device 120 transmits the merchantbeacon identifier to the user computing device 110 via a BLE, Bluetooth,Wi-Fi, or NFC wireless communication network 140. In this example, theuser computing device 110 retransmits the received merchant beaconidentifier to the account management system 160 along with a user 101account identifier identifying the user 101.

In block 920, the account management system 160 determines whether a newuser 101 is in range of the merchant beacon device 120. For example, ifthe account management system 160 receives a new user 101 accountidentifier in addition to the same merchant beacon identifier, theaccount management system 160 may determine that a new user 101 is inrange of the merchant beacon device 120. In this example, the accountmanagement system 160 may infer that the new user 101 has entered themerchant location based on receipt of the new user 101 accountidentifier. In another example, if the account management system 160does not receive any new user 101 account identifiers along with thesame merchant beacon identifier within a threshold length of time, theaccount management system 160 may determine that no new users 101 haveentered the network 140 range of the merchant beacon device 120.

If a new user 101 is in range of the merchant beacon device 120, themethod 870 proceeds to block 810 in FIG. 8. For example, the accountmanagement system 160 receives a new user 101 account identifier inaddition to the same merchant beacon identifier. In this example, theaccount management system 160 infers that the new user 101 has enteredthe merchant location based on receipt of the new user 101 accountidentifier and the same merchant beacon identifier as previouslyreceived from the first user 101.

Returning to FIG. 8, according to the methods previously discussed, inblock 810, the account management system 160 extracts a facial templateassociated with the new user 101 account identifier. In another example,the account management system 160, in addition to or instead of thefacial template associated with the new user 101, the account managementsystem 160 extracts an audio template and/or a challenge and responseassociated with the new user 101. In an example embodiment, the accountmanagement system 160 transmits the facial template, audio template,and/or challenge and response to the one or more merchant POS devices130. The one or more merchant POS devices 130 adds the new user's 101facial template, audio template, and/or challenge and response to thecustomer log according to the example method previously described inmethod 250 in FIG. 8.

Returning to FIG. 9, in block 920, if there is a not a new user 101 inrange of the merchant beacon device 120, the method 870 proceeds toblock 930. For example, the account management system 160 does notreceive any new user 101 account identifiers along with the samemerchant beacon identifier within a threshold length of time anddetermines that no new users 101 have entered the network 140 range ofthe merchant beacon device 120.

In block 930, the merchant POS device 130 determines whether a currentuser 101 has moved out of range of the merchant beacon device 120. In anexample embodiment, the user computing device 110 continues to receivethe merchant beacon identifier from the merchant beacon device 120, andretransmits the merchant beacon identifier along with the user 101account identifier to the account management system 160. In this exampleembodiment, the user computing device 110 periodically transmitsinformation comprising the merchant beacon identifier and user 101account identifier to the account management system 160, as long as theuser computing device 110 continues to detect the merchant beacon device120 and/or receives the merchant beacon device 120 identifier viaperiodic scans. For example, the user computing device scans for themerchant beacon device 120 every five seconds. In another exampleembodiment, the user computing device 110 periodically transmitsinformation comprising the merchant beacon identifier and user 101account identifier to the account management system 160, as long as theuser computing device 110 maintains a wireless network 140 connectionwith the merchant beacon device 120. For example, the user computingdevice 110 transmits this information to the account management system160 at every five seconds. In this example embodiment, if the accountmanagement system 160 ceases to receive the information from the usercomputing device for a predefined number of intervals, the accountmanagement system 160 may determine that the corresponding user 101 hasmoved out of range of the merchant beacon device. In this exampleembodiment, if the account management system 160 continues to receivethe information transmitted by the user computing device 110 at theexpected intervals, the account management system 160 determines thatthe user 101 is still in network 140 range of the merchant beacon device120.

If no current user 101 has moved out of range of the merchant beacondevice 120, the method 870 proceeds to block 250 in FIG. 2. For example,the account management system 160 continues to receive the merchantbeacon identifier and user 101 account identifier transmitted by theuser computing device 110 at the expected intervals and determines thatthe user 101 is still in network 140 range of the merchant beacon device120.

Returning to FIG. 9, in block 930, if a current user 101 has moved outof range of the merchant beacon device 120, the method 870 proceeds toblock 940.

In block 940, the merchant POS device 130 receives a notification fromthe account management system 160 that a current user 101 is out ofnetwork range of the merchant beacon device 120. In another exampleembodiment, the merchant POS device 130 receives a notification from theaccount management system 160 that the user computing device 110associated with the current user 101 has stopped sending notificationsto the account management system 160 comprising the merchant beacondevice 120 identifier. For example, the merchant POS device 130 receivesthe user 101 account identifier for the current user 101 associated witha user computing device 110 that is either out of network range or hasstopped transmitting notifications comprising the merchant beacon device120 identifier to the account management system 160. The merchant POSdevice 130 accesses the customer log and finds an entry corresponding tothe current user 101. For example, the customer log is maintained by theaccount management system 160 and the merchant POS device 130 accessesthe customer log over the network 140 by communicating with the accountmanagement system 160. In another example embodiment, the accountmanagement system 160 does not transmit a notification to the merchantPOS device 130 that the current user 101 is out of network range. Inthis example embodiment, the account management system 160 accesses thecustomer log and deletes the facial template of the current user 101from the customer log. In another example embodiment, in addition todeleting the facial template of the current user 101 from the customerlog, the merchant POS device 130 deletes the audio template and/or thechallenge and response associated with the current user 101 from thecustomer log.

In block 950, the merchant POS device 130 deletes the facial template ofthe current user 101 from the customer log. For example, the customerlog comprises a table, and the merchant POS device 130 deletes orrequests the deletion of an entry or row corresponding to dataassociated with the current user 101 for which the POS device 130received the notification. In another example, the merchant POS device130 deletes the associated audio template and/or associated challengeand response of the current user 101 from the customer log, in additionto the facial template of the current user 101. In another exampleembodiment, the account management system 160 accesses the customer logand deletes the facial template, audio template, and/or challenge andresponse of the current user 101 from the customer log.

From block 950, the method 870 proceeds to block 250, in FIG. 2.

Returning to FIG. 2, in block 250, the user 101 initiates a transactionat the merchant POS device 130. The method for initiating a transactionat a merchant POS device 130 is described in more detail hereinafterwith reference to the method 260 described in FIG. 10. In the exampleembodiments described herein, the user 101 initiates a “hands freetransaction” at the merchant POS device 130. An example hands freetransaction does not require any interaction with the user computingdevice 110 on the part of the user 101. In another example, a hands freetransaction requires only minimal interaction with the user computingdevice 110 by the user 101.

FIG. 10 is a block diagram depicting a method 250 for initiating, by auser 101, a transaction at a merchant POS device 130, in accordance withcertain example embodiments. The method 250 is described with referenceto the components illustrated in FIG. 1.

In block 1010, the user 101 approaches the merchant POS device 130. Inan example embodiment, at a time prior to approaching the merchant POSdevice 130, the user 101 browses the merchant location and selects oneor more items to purchase. In this example embodiment, the user 101 maycollect the one or more items and carry, or otherwise transport viaphysical basket or shopping cart, the one or more items to the merchantPOS device 130.

In block 1020, the merchant POS device 130 operator 102 totals the itemsof the user 101 for purchase. In an example embodiment, the merchant POSdevice operator 102 scans barcodes attached to the one or more items orotherwise enters descriptions and prices associated with the one or moreitems into the merchant POS device 130. In an example embodiment, afterscanning or manually entering the items into the merchant POS device130, the merchant POS device operator 102 actuates an object on the userinterface 135 of the merchant POS device 130 to order the merchant POSdevice 130 to total the items. In an example embodiment, the merchantPOS device 130 displays, via the user interface 135, the total to theuser 101.

In block 1030, the merchant POS device 130 operator asks the user 101 toselect a payment option. In an example embodiment, the merchant POSdevice 130 displays one or more payment options that the user 101 mayselect to use in a transaction. Example payment options comprise paymentvia a application 113 associated with the account management system 160,payment by cash, payment by check, payment by credit card, payment bydebit card, and/or any other means of payment that the merchant systemcan or is willing to accept for payment from the user 101. In an exampleembodiment, the one or more payment options are displayed as objects onthe user interface 135 and are selectable by the merchant POS deviceoperator 102 in response to the user 101 directing the merchant POSdevice 102 operator to make a selection.

In block 1040, the user 101 directs the merchant POS device operator 102to initiate a transaction via the application 113. In an exampleembodiment, in response to receiving a verbal request from the user 101to select the application 113 as a payment option, the merchant POSdevice operator 102 actuates an object on the user interface 135 of themerchant POS device 130 corresponding to the application 113 paymentoption.

In block 1050, the merchant POS device operator 102 selects an option onthe merchant POS device 130 to initiate a transaction using theapplication 113. In an example embodiment, the merchant POS device 130displays a confirmation screen after the merchant POS device operator102 selects an option to initiate a transaction using the application113. An example confirmation screen may display information summarizingthe potential transaction and comprising one or more of a transactiontotal, a description of the one or more items being purchased by theuser 101, and an indication that the user 101 selected the application113 as the method of payment for the transaction. An exampleconfirmation screen may further display options to confirm thetransaction or cancel the transaction. In an example embodiment, theuser 101 reviews the confirmation screen, determines that theinformation displayed on the confirmation screen is correct, determinesto continue with the transaction, and directs the merchant POS deviceoperator 102 to select the option to confirm the transaction via theuser interface 135.

From block 1050, the method 250 proceeds to block 260 in FIG. 2.

Returning to FIG. 2, in block 260, the merchant POS device 130identifies the user 101 via facial recognition. The method foridentifying a user 101 via facial recognition is described in moredetail hereinafter with reference to the method 260 described in FIG.11. In other example embodiments, the merchant POS device 130 identifiesthe user 101 via audio recognition and/or via a challenge and response.

FIG. 11 is a block diagram depicting a method 260 for identifying, by amerchant POS device 130, a user 101 via facial recognition, inaccordance with certain example embodiments. The method 260 is describedwith reference to the components illustrated in FIG. 1.

In block 1110, the merchant POS device 130 captures video of the user101. In an example embodiment, in response to receiving a request toidentify the user 101, the merchant POS device 130 activates the camera132 to begin to capture a video of the surroundings of the merchant POSdevice 130. In an example embodiment, the merchant POS device 130captures a video feed of the user's 101 face. In another exampleembodiment, the camera 132 continuously captures, but does not record, avideo feed of its surroundings. In this example embodiment, when themerchant POS device 130 receives an input from the merchant POS device130 operator 102, a request to identify the user 101 is received fromthe account management system 160. The camera 132 begins to record thevideo feed for a threshold amount of time. In an example embodiment, theuser 101 may be moving during the period in which the camera 132 recordsthe video feed. In an example embodiment, the camera 132 extracts afacial image by determining a particular frame of the video feed andarea of the instance of the video feed corresponding to the face of theuser. In another example embodiment, the merchant POS device 130captures one or more facial images of the user 101 via the camera 132.

In block 1120, the camera 132 extracts a facial image of the user 101from the captured video. In an example embodiment, the camera 132determines a frame of the captured video to provide an image of theuser's 101 face and extracts the frame of the captured video.

In certain other example embodiments, the camera 132 determines a frameof the captured video to provide an image of the faces of a plurality ofusers 101. For example, the frame comprises an image of the face of afirst user 101, a second user 101, and a third user 101 at differentlocations in the image. In this example, one camera 132 associated witha particular merchant POS device 130 captures video of an environmentcorresponding to an area in the proximity of multiple merchant POSdevices 130. In this example embodiment, the camera 132 determines towhich particular merchant POS device 130 each of the plurality of facesof the corresponding plurality of users 101 in the extracted image.

In block 1130, merchant POS device 130 generates a facial template fromthe captured facial image. In another example, the merchant POS device130 generates the facial template. In an example, the facial template isof a predetermined size, for example, a 128-byte facial template. In anexample, the account management system 160 generates a facial templatecomprising a computer code representation of the digital facial image.For example, the facial template may describe key features of the facialimage of the user 101, such as shape, color, line, value, space, form,texture, or other useful or relevant feature of the image or ofparticular regions of the image. In another example, the facial templateis generated by processing the facial image through a convolutionalneural network. In an example embodiment, the merchant POS device 130stores the generated facial template in a data storage unit associatedwith the merchant POS device 130. For example, the merchant POS device130 may comprise a log of facial templates of current customers.

In certain other example embodiments, the merchant POS device 130continuously captures a video feed of its surroundings as users 101enter and leave the vicinity of one or more merchant POS devices 130over the course of a certain time period. In this example embodiment,the merchant POS device 130 and/or camera 132 is able to continuouslymonitor the incoming video feed to detect faces from extracted frames ofthe video feed. In this example embodiment, the camera 132, each timethe camera 132 detects the presence of one or more faces in the videofeed, the camera 132 extracts a frame of the video feed comprising oneor more facial images of one or more corresponding detected faces andcreates facial templates based on the extracted one or more facialimages. In this example embodiment, the merchant POS device 130 storesfacial templates in the log of facial templates of current customers asthey are generated. In this example embodiment, as the camera 132 or themerchant POS device 130 generates subsequent facial templates. Themerchant POS device 130 determines whether a generated subsequent facialtemplate is similar to or within a predefined threshold to any of thefacial templates already stored in the customer log. If the generatedsubsequent facial template is similar to or within a threshold to any ofthe facial templates already stored in the customer log, the merchantPOS device 130 adds the facial template to the customer log andassociates the facial template to one or two particular merchant POSdevices 130 based on the position of the associated facial images in theextracted frame of the captured video. In an example embodiment, at thetime that the merchant beacon device 120 and/or merchant POS devices 130are installed at the merchant location, the account management system160 associates the merchant beacon device 120 with one or moreparticular merchant POS devices 130. In this example embodiment, theaccount management system 160 comprises a database comprising a tablethat correlates the merchant beacon device 120 code associated with themerchant beacon device 120 with device identifiers associated with theone or more merchant POS devices 130 associated with the merchant beacondevice 120. In an example embodiment, when the account management system160 receives the merchant beacon device 120 code retransmitted form theuser computing device 110, the account management system 160 determinesthe one or more POS devices 130 associated with the merchant beacondevice 120 identified by the merchant beacon device 120 code by lookingup the merchant beacon device 120 code in the table that associatesmerchant beacon device 120 codes with associated merchant POS device 130identifiers. In an example embodiment, if the generated subsequentfacial template is not similar to within a threshold to any facialtemplates already stored in the customer log, the merchant POS device130 deletes, ignores, and/or does nothing with the generated facialtemplate. In this example embodiment, if the merchant POS device 130determines that a certain facial image is no longer in the field of thevideo feed, the corresponding facial template is deleted from the log offacial templates of current customers.

In block 1140, the merchant POS device 130 deletes the captured videoand the extracted facial image. For example, the camera 132 does notstore captured images or video. In this example, facial templatesgenerated by the camera 132 comprise computer code representations offacial images of users 101. In this example, after generating a facialtemplate or after a threshold time has passed after capturing video orimages or extracting an image from a video, the merchant POS device 130deletes any captured or extracted video or images.

In block 1150, the merchant POS device 130 retrieves facial templatesfrom the customer log. For example, the customer log comprises facialtemplates received from the account management system 160 correspondingto all users 101 whose associated user computing devices 110 are locatedwithin a network distance of a merchant beacon device 120.

In block 1160, the merchant POS device 130 compares the generated facialtemplate from captured facial image to facial templates from thecustomer log. In another example embodiment, the account managementsystem 160 compares the generated facial template from captured facialimage to facial templates from the customer log. Comparing the generatedfacial template generated from the captured facial image to facialtemplates from the customer log may comprise comparing one or moreindividual features or components of the templates. For example, thefacial template may describe key features of the facial image of theuser 101, such as shape, color, line, value, space, form, texture, orother useful or relevant feature of the image or of particular regionsof the image.

In block 1170, the merchant POS device 130 determines whether there is amatch between the generated facial template and one of the facialtemplates from the customer log. In an example, a match comprises acomplete match between the generated facial template and a facialtemplate from the current customer log. In another example, a matchcomprises a similarity greater than a predefined threshold similaritybetween the generated facial template and a facial template from thecurrent customer log. For example, the merchant POS device 130determines a similarity value for each of the one or more individualfeatures or components between the generated facial template and thefacial template from the customer log. In this example, the merchant POSdevice 130 may then determine an overall similarity value between thegenerated facial template and the facial template from the customer logand compare the overall similarity value against the predefinedthreshold similarity value. If the overall similarity value is greaterthan or equal to the predefined threshold similarity value, the merchantPOS device 130 may determine that there is a match between the generatedfacial template and the facial template from the customer log. However,if the overall similarity value is less than the predefined thresholdsimilarity value, the merchant POS device 130 may determine that thereis no match between the generated facial template and the facialtemplate from the customer log.

In certain other example embodiments, the account management system 160compares the generated facial template against an existing facialtemplate associated with the user 101 account. In an example embodiment,the account management system 160 receives a user 101 account identifierfrom the merchant POS device 130 that is associated with the user 101account and identifies the user 101 account based on this user 101account identifier. In this example embodiment, the account managementsystem 160 extracts the existing facial template from the user 101account associated with the received user 101 account identifier. In anexample embodiment, the account management system 160 compares likefeatures from the generated facial template against corresponding likefeatures from the existing facial template associated with the user 101account. For example, the account management system 160 may compare thekey features of the generated facial template, such as shape, color,line, value, space, form, texture, or other useful or relevant featureof the image or of particular regions of the image against similar keyfeatures of the existing facial template associated with the user 101account. In an example embodiment, for each key feature, the accountmanagement system 160 generates an individual delta value that describesa degree or proportion of difference between the generated facialtemplate and the existing facial template. In this example embodiment,the account management system 160 generates an overall delta value oraggregated delta value based on each individual delta value. Forexample, the overall delta value or aggregated delta value gives anoverall measure of the degree of difference between the generated facialtemplate and the existing facial template. In this example, anindividual delta value or overall (or aggregated) delta value maycomprise a number between 0 and 1, with 1 meaning that the facialtemplates are completely alike and 0 meaning that the facial templatesare completely unalike.

If a facial template from the customer log matches the generated facialtemplate, the method 260 proceeds to block 290 in FIG. 2. For example,the merchant POS device 130 processes the transaction.

Returning to block 1170, if none of the facial templates from thecustomer log matches the generated facial template, the method 260proceeds to block 270 in FIG. 2.

Returning to block 270, in FIG. 2, the merchant POS device 130identifies the user 101 via voice recognition. In another exampleembodiment, the account management system 160 does not identify the user101 via voice recognition and proceeds to identify the user 101 viachallenge and response. For example, the merchant POS device 130 and/oraccount management system 160 is unable to identify the user via facialrecognition and/or if the merchant POS device 130 and/or accountmanagement system 160 identify two or more users 101 via facialrecognition. The method for identifying, by a merchant POS device 130, auser 101 via voice recognition is described in more detail hereinafterwith reference to the method 270 described in FIG. 12. In anotherexample embodiment, the account management system 160 identifies theuser 101 via voice recognition. Accordingly, one or more functionsherein described as being performed by the merchant POS device 130 mayalso be performed by the account management system 160.

FIG. 12 is a block diagram depicting a method 270 for identifying, by amerchant POS device 130, a user 101 via voice recognition, in accordancewith certain example embodiments. The method 270 is described withreference to the components illustrated in FIG. 1.

In block 1210, the merchant POS device 130 determines that the user 101cannot be identified via the facial template. In certain exampleembodiments, it may be necessary to identify a user 101 via voicerecognition because the merchant POS device 130 is not able to identifythe user 101 based on facial recognition. For example, the merchantcamera 132 may be unable to extract an adequate facial image of the user101 from the video feed to generate a facial template. In anotherexample, the merchant POS device 130 is unable to find a match for thefacial template of the user 101 in the customer log. In yet anotherexample, the merchant POS device 130 identifies a matching user 101facial template in the customer log, however, the identified user 101facial template is assigned to two adjacent merchant POS devices 130. Inthis example, the user 101 may have to identify himself via voicerecognition at the POS device 130 at which the user 101 initiates thetransaction.

In block 1220, the merchant POS device 130 retrieves audio templatescorresponding to users 101 from the customer log. As previouslydiscussed, when a user 101 establishes an account with the accountmanagement system 160, the user 101 may submit an audio recording of theuser's 101 voice to the merchant POS device 130. In this example theaccount management system 160 establishes an audio templatecorresponding to the user 101 based on the received audio recording. Inanother example embodiments, the user 101 does not submit an audiorecording to the account management system 160 at the time the user 101establishes the user 101 account. In this other example embodiment, theaccount management system 160 does not have an audio template associatedwith the user 101 account and cannot verify the user 101 via voicerecognition.

In block 1230, the merchant POS device 130 records audio of the user101. For example the merchant system POS device 130 displays the requestfor the user 101 to record audio. For example, the merchant system POSdevice 130 may display directions to the user 101 to record an audiorecording. For example, the user 101 may be directed to speak the samewords in the same intonation as the user 101 did when establishing theaudio template with the account management system 160 at the time ofsetting up the user 101 account. In an example, the merchant system POSdevice 130 records a voice input of the user 101. For example, themerchant POS device operator 102 may actuate an object on the userinterface 135 to activate an audio module 131 to receive an audio inputof the user 101. In an example, the merchant POS device 130 or accountmanagement system 160 generates an audio template based on the receivedvoice input. In an example, the audio template comprises a computer coderepresentation of one or more features of the voice input.

In block 1240, the merchant POS device 130 compares the received voiceinput against the retrieved audio templates corresponding to users 101from the customer log. In an example embodiment, the account managementsystem 160 receives the voice input from the merchant POS device 130 andcompares the received voice input against retrieved audio templatescorresponding to users 101 from the customer log.

In block 1250, the merchant POS device 130 determines whether there is amatch between the received voice input and one of the retrieved audiotemplates from the customer log. In an example embodiment, the merchantPOS device 130 compares one or more features between the received voiceinput and each of the retrieved audio templates from the customer log.In an example embodiment, if the similarity between the received voiceinput and a particular audio template exceeds a predefined threshold,the merchant POS device 130 determines that the received voice inputmatches the particular audio template. In another example embodiment, ifthe similarity between the received voice input and the particular audiotemplate is less than the predefined threshold, the merchant POS device130 determines that the received voice input does not match theparticular audio template. Comparing the generated audio templategenerated from the captured voice input to audio templates from thecustomer log may comprise comparing one or more individual features orcomponents of the templates. For example, the facial template maydescribe key features of the voice input of the user 101, such asvolume, pitch, or other useful or relevant feature of the audiotemplate.

In an example, a match comprises a complete match between the generatedaudio template and a facial template from the current customer log. Inanother example, a match comprises a similarity greater than apredefined threshold similarity between the generated facial templateand a facial template from the current customer log. For example, themerchant POS device 130 determines a similarity value for each of theone or more individual features or components between the generatedaudio template and the audio template from the customer log. In thisexample, the merchant POS device 130 may then determine an overallsimilarity value between the generated audio template and the audiotemplate from the customer log and compare the overall similarity valueagainst the predefined threshold similarity value. If the overallsimilarity value is greater than or equal to the predefined thresholdsimilarity value, the merchant POS device 130 may determine that thereis a match between the generated audio template and the audio templatefrom the customer log. However, if the overall similarity value is lessthan the predefined threshold similarity value, the merchant POS device130 may determine that there is no match between the generated audiotemplate and the audio template from the customer log.

If an audio template from the customer log matches the received voiceinput, the method 270 proceeds to block 290 in FIG. 2. For example, thesimilarity between a particular audio template from the customer log andthe received audio recording of the user 101 exceeds a predefinedthreshold and the merchant POS device 130 determines that there is amatch.

Returning to block 1250, if none of the audio templates from thecustomer log match the received voice input, the method 270 proceeds toblock 280 in FIG. 2. In another example embodiment, the merchant POSdevice 130 is unable to receive an audio recording of the user 101, thereceived audio recording of the user 101 is inadequate, or the receivedaudio recording of the user 101 cannot be used to identify the user 101.

Returning to FIG. 2, in block 280, the user 101 is identified by themerchant system POS operator 102 via a challenge and a response. Themethod for identifying a user 101 via a challenge and a response isdescribed in more detail hereinafter with reference to the method 280described in FIG. 13.

FIG. 13 is a block diagram depicting a method 280 for identifying, by amerchant POS device operator 102, a user 101 via a challenge and aresponse, in accordance with certain example embodiments. The method 280is described with reference to the components illustrated in FIG. 1.

In block 1310, the POS device operator 102 issues a challenge to theuser 101. In an example, the merchant POS device operator 102 asks theuser 101 for the initials of the user 101. In another example, themerchant POS device operator 102 asks the user 101 for the last fourdigits of the phone number of the user 101. In another example, themerchant POS device operator 102 asks the user 101 for a configuredpassword. Any suitable challenge may be issued by the merchant POSdevice operator 102. In an example embodiment, the response to thechallenge does not provide any secure or private information.

In block 1320, the user 101 provides a challenge response. As describedin the previous example, the responses may be the initials of the user101, the last four digits of the phone number of the user 101, or aconfigured password. Any configured challenge response may be utilized.In certain embodiments, the response may be a spoken response, a handgesture, a keypad entry, a display of an identification card, or anysuitable response.

In block 1330, the POS device operator 102 inputs the response into themerchant POS device 130. In an example, if the user 101 indicates thatthe initials of the user 101 are “AC,” then the merchant POS deviceoperator 102 inputs “AC” into the application 133 of the merchant POSdevice 130. In an example, the user interface 135 of the merchant POSdevice 130 displays a request for an entry of the response of the user101. The merchant POS device operator 102 enters the response via avirtual or physical keyboard, voice dictation, or in any suitablemanner. In an alternate example, the user 101 enters the response intothe user interface 135 of the merchant POS device 130.

In block 1340, the merchant POS device 130 displays potential users 101based on the challenge response. The merchant POS device 130 displayspotential users 101 based on the challenge response. A list of users 101that are associated with the challenge response are displayed on themerchant POS device 130 to the merchant POS device operator 102. Forexample, if ten customers are in the vicinity of the merchant beacondevice 120, then the merchant POS device 130 may have received from theaccount management system 160 a challenge response associated with eachof the respective ten customers. When the merchant POS device 130receives the challenge response input, only the potential users 101 thatare associated with the challenge response are displayed to the merchantPOS device operator 102.

In the example, if the merchant POS device operator 102 inputs “AC” asthe initials of the user 101 associated with the transaction, then onlythe potential users 101 with those initials will be displayed to themerchant POS device operator 102 by the application 133. The application133 accesses a database on the account management system 160 or anothercomputing device and identifies the initials of the potential users 101that have provided tokens. The application 133 identifies the one ormore potential users 101 that have the initials “AC” and displays theidentified user 101 accounts to the merchant POS device operator 102. Inthe example, two of the ten customers that are in the vicinity of themerchant beacon device 120 have the initials “AC.” The user 101 accountsof the two customers are displayed to the merchant POS device operator102.

In certain example embodiments, all of the nearby customers who have hadtokens transmitted to the merchant POS device 130 are presented to themerchant POS device operator 102 and the merchant POS device operator102 selects the appropriate user 101 account.

In another embodiment, the merchant POS device 130 or the accountmanagement system 160 presents additional challenges until there is asingle matching user 101 remaining.

The application 133 may display a picture of the potential user 101accounts that are presented to the merchant POS device operator 102. Forexample, each user 101 may associate a picture with a user 101 account.When the merchant POS device 130 presents the one or more potential user101 accounts to the merchant POS device operator 102, the merchant POSdevice operator 102 may select the appropriate user 101 account based onthe picture matching the user 101 conducting the transaction. Otheridentifying information may be presented instead of, or in addition to,a picture. For example, the name of the user 101 may be displayed andthe merchant POS device operator 102 may identify the potential user 101with that name. Any other suitable identifying information may bepresented.

In block 1350, the merchant POS device operator 102 selects the user 101account for use in a transaction. After identifying the displayedpicture of the user 101, the merchant POS device operator 102 may inputa selection of the user 101 by actuating a user interface 135 controlassociated with the picture, or by inputting the selection in anysuitable manner. If the picture doesn't match any of the potentialusers, then the merchant POS device operator 102 may cancel thetransaction, notify the user 101 of the discrepancy, or perform anyother suitable action.

In an example, only a single user 101 account is presented in the listof potential users 101. If only a single user 101 account is identified,then the method may proceed after the merchant POS device operator 102verifies that the displayed picture matches the user 101. If the picturedoesn't match, then the merchant POS device operator 102 may cancel thetransaction, notify the user 101 of the discrepancy, or perform anyother suitable action.

From block 1350, the method 280 proceeds to block 290 in FIG. 2.

Returning to FIG. 2, in block 290, a transaction is processed. Themethod for processing a transaction is described in more detailhereinafter with reference to the method 290 described in FIG. 14.

FIG. 14 is a block diagram depicting a method 290 for processing atransaction, in accordance with certain example embodiments. The method290 is described with reference to the components illustrated in FIG. 1.

In block 1410, the merchant POS device 130 generates a paymentauthorization request based on the payment token and other transactioninformation. In an example embodiment, the payment authorization requestcomprises the payment token received from the account management system160 for the user 101 along with transaction details. Example transactiondetails comprise a transaction total, a description of one or more itemsbeing purchased, a merchant identifier, a merchant payment accountidentifier, and/or other relevant transaction details.

In block 1420, the merchant POS device 130 transmits the paymentauthorization request to the issuer system 150. For example, themerchant POS device 130 communicates the payment authorization requestto the issuer system 150 via the network 140.

In block 1430, the issuer system 150 approves the payment authorizationrequest. In an example embodiment, the issuer system 150 identifies theuser payment account based on the received payment token. For example,the issuer system 150 accesses a database that associates payment tokenswith user 101 payment account identifiers. In an example, the databasefurther associates payment tokens with one or more conditions, such as alength of time for which the payment token is valid. For example, apayment token may only be valid for a threshold length of time, forexample one hour, after it is generated by the account management system130. In this example, as part of the transaction details in the paymentauthorization request, a current timestamp is received from the merchantPOS device 130. The issuer system 150 compares the timestamp to the oneor more time conditions described in the database and associated withthe payment token or associated with other data received from theaccount management system 160.

In another example, the payment token is valid only for use at aparticular merchant system. In this example, the transaction detailsreceived with the payment authorization request from the merchant POSdevice 130 comprise a merchant system identifier. In this example, theissuer system 150 determines that the payment token is valid if themerchant identifier received in the transaction details matches themerchant identifier identified in the one or more conditions associatedwith the payment token in the database.

In certain other example embodiments, other conditions related to time,location, merchant identifier, or a combination of these conditionsand/or other conditions may be specified in the database as associatedwith one or more particular payment tokens. In an example embodiment,the issuer system 150 verifies that a payment token received as part ofa payment authorization request is valid based at least in part on datareceived from the merchant POS device 130 and/or data currentlyavailable to the issuer system 150. In an example embodiment, to processthe transaction, the issuer system 150 identifies the user paymentaccount associated with the received payment token in the database andprocesses the transaction using the transaction details and theidentified user payment account information.

In block 1440, the merchant POS device 130 receives an approval of thepayment authorization request from the issuer system 150. In an exampleembodiment, the issuer system 150 either approves or declines thepayment authorization request. In this example embodiment, the issuersystem 150 may base the decision of whether to approve or decline thepayment authorization request based on a total amount of transaction thecurrent available credit of the user 101 for the user 101 paymentaccount. In an example embodiment, the merchant POS device 130 receives,via the network 140, the approval of the payment authorization requestfrom the issuer system 150 if the issuer system 150 approves the paymentauthorization request. In another example embodiment, the merchant POSdevice 130 receives a notice of declined payment authorization requestfrom the issuer system 150 via the network 140 if the issuer system 150declines the payment authorization request.

In block 1450, the merchant POS device 130 displays a confirmation ofthe approved transaction to the user 101. An example confirmation of theapproved transaction comprises a total amount charged to the user 101payment account, an identification of the user 101 payment account, amerchant system name, and/or other relevant or useful information. Inanother example embodiment, the merchant POS device 130 displays anotification of a declined transaction in response to receiving a noticeof declined payment authorization request from the issuer system 150.For example, the merchant POS device 130 displays a message reading“This transaction has been declined” to the user via the user interface135 of the merchant POS device 130. In another example embodiment, themerchant POS device 130 prints a receipt for the user 101.

In certain example embodiments, from block 1450, the method 290 proceedsto block 295 in FIG. 2. In certain other example embodiments, the method290 may end at block 1450. In certain example embodiments, the user 101does not process a transaction.

Returning to FIG. 2, in block 295, a user 101 updates user 101 accountdata associated with an account of the user 101. For example, at a timeafter initially configuring user account data, the user 101 associatedwith the user 101 account may wish to update the user's 101 accountdata. Example account data comprises one or more of payment accountdata, user 101 contact information, user 101 demographic data, user 101permission settings for hands-free transactions, the facial template,the audio template, the challenge and response, and/or any other user101 account data associated with the user 101 account. In an exampleembodiment, updating user 101 account data may comprise adding,deleting, editing, rearranging, or other appropriate action with respectto user 101 account data. For example, the user 101 may desire to addnew payment account information associated with a credit account or bankaccount of the user 101. In another example, the user 101 may desire toupdate the facial template, audio template and/or challenge and responseat a time before an initial hands-free transaction or at a time afterthe initial hands-free transaction and before a subsequent hands-freetransaction.

The method for updating user 101 account data at a merchant POS device130 via a password comprising a facial template is described in moredetail hereinafter with reference to the method 295 described in FIG.15.

FIG. 15 is a block diagram depicting a method 295 for updating user 101account data at a merchant POS device 130 via a password comprising afacial template, in accordance with certain example embodiments. Themethod 295 is described with reference to the components illustrated inFIG. 1. In another example embodiment described herein, the user 101 mayrequest to update user 101 account data via the user computing device110 or other computing device using a password comprising a facialtemplate of the user 101. In another example embodiment, the user 101may request to update user 101 account data via the user computingdevice 110 or other computing device using a password comprising anaudio template. Accordingly, one or more of the functions described asbeing performed by the merchant POS device 130 with respect to themethod for updating user 101 account data may be performed by the usercomputing device 110.

In block 1510, the user 101 requests to change user 101 account data atthe merchant POS device 130. In an example embodiment, the user 101 hascompleted a hands-free transaction at the merchant location and nowdesires to update user 101 account data. In another example embodiment,the user 101 has not initiated a transaction at the merchant locationand desires to update user 101 account information. In yet anotherexample embodiment, the user 101 is in the process of a transaction atthe merchant location and desires to update user 101 accountinformation. For example, the user 101 has been identified and themerchant POS device 130 has received the user 101 account informationcomprising payment account information. In this example, the user 101may not be satisfied with his payment account options or may otherwisedesire to update his user 101 account information at that point. Incertain example embodiments, the user 101 provides a user 101 accountnumber associated with the user 101 account to the merchant POS device130 operator to input to the merchant POS device 130, taps the usercomputing device 110 to a reader of the merchant POS device 130 toprovide account information comprising a user 101 account identifier, orotherwise provides a user 101 account identifier to the merchant POSdevice 130.

In block 1515, the merchant POS device 130 operator 102 selects anoption to update user 101 account data. In an example embodiment, themerchant POS device 130 operator 102 selects an option to update user101 account data as directed by the user 101. In another exampleembodiment, the user 101 selects the option to update the user 101account data. In an example embodiment, selecting an option to updateuser 101 account data comprises actuating one or more objects on theuser interface of the merchant POS device 130. For example, the merchantPOS device 130 displays, via the user interface, a user interface objectthat reads “update user account information.”

In block 1520, the merchant POS device 130 captures a facial image ofthe user 101. In an example embodiment, the merchant POS device 130captures the facial image of the user 101 in response to receiving aninput of a selection of a user interface object requesting the merchantPOS device 130 to update user 101 account information. In this exampleembodiment, in response to the user actuating the object on the userinterface 115, the camera 132 receives a command from the application133 to capture an image of the user 101. In another example embodiment,the camera 132 receives a command from the application 133 to capture aplurality of images of the user 101. For example, each of the pluralityof images of the user 101 may correspond to a particular pose of theuser's 101 face. An example facial image may comprise a digital image ofthe face of a user 101. In an example embodiment, the account managementsystem 160 may establish guidelines for users 101 in submitting facialimages. For example, the user 101 may be required to remove any hats,head coverings, glasses, or other objects or accessories that mayocclude regions of the user's 101 face so that application 133 mayreceive a complete depiction of the user's 101 face. In an exampleembodiment, instead of directly capturing the user 101 facial image, themerchant POS device 130 may be communicatively coupled via a network 140to a merchant camera device, which captures the image of the user 101.In certain example embodiments, the merchant POS device 130 operator oruser 101 must actuate one or more objects on the user interface of themerchant POS device 130 to direct the merchant POS device 130 or cameradevice to capture the facial image of the user 101.

In block 1530, the merchant POS device 130 transmits the captured facialimage to the account management system 160. In an example embodiment,the merchant POS device 130 transmits the captured facial image to theaccount management system 160 via the network 140. In another exampleembodiment, the merchant camera device transmits the captured image tothe account management system 160 via the network 140 or otherwisecommunicates the captured image to the merchant POS device, wherein themerchant POS device 130 transmits the captured image to the accountmanagement system 160 via the network 140. In an example embodiment, themerchant POS device 130 transmits the captured facial image to theaccount management system 160 along with the user 101 account identifierand a request to update user 101 account information to the accountmanagement system 160 via the network 140. In another exampleembodiment, the merchant POS device 130 already captured a facial imageof the user 101 in a recent transaction and does not capture asubsequent facial image of the user 101 but transmits to the accountmanagement system 160 the previously captured facial image of the user101 associated with the recent transaction.

In certain example embodiments, the merchant POS device 130 or merchantcamera device determines if the captured facial image is a valid facialimage or an invalid facial image. For example, a valid facial imagecomplies with guidelines predetermined by the account management system160 and an invalid facial image does not comply with one or more of theguidelines. For example, if the merchant POS device 130 or merchantcamera device captures a facial image that comprises incorrectdimensions, if part or all of the user's 101 face is occluded, or if theimage is too dark or too bright, the merchant POS device 130 or merchantcamera device rejects the invalid facial image and displays a requestdirecting the user 101 to capture a subsequent facial image. In anotherexample, the captured facial image that is not of a face and theapplication 133 or camera device determines, via facial recognition,that the image is not of a face and directs the camera of the merchantPOS device 130 or merchant camera device to capture a subsequent facialimage of the user 101 to transmit to the account management system 160.In this example, the merchant POS device 130 or merchant camera devicecaptures a subsequent facial image via the user computing device 110. Inanother example, the account management system 160 determines if thecaptured facial image is a valid facial image or an invalid facialimage. Accordingly one or more functions described herein as beingperformed by the merchant POS device 130 may also be performed by theaccount management system 160.

In block 1535, the account management system 160 generates a facialtemplate based on the captured facial image. In an example embodiment,the account management system 160 creates a facial template associatedwith the user 101 account based on the received facial image. In anotherexample embodiment, the account management system 160 generates acorresponding facial template for each of a plurality of received facialimages associated with the user 101 account. In an example, the facialtemplate is of a predetermined size, for example, a 128-byte facialtemplate. In an example, the account management system 160 generates afacial template comprising a computer code representation of the digitalfacial image. For example, the facial template may describe key featuresof the facial image of the user 101, such as shape, color, line, value,space, form, texture, or other useful or relevant feature of the imageor of particular regions of the image. In an example, the facialtemplate is generated by processing the facial image through aconvolutional neural network.

In another example embodiment, after the merchant POS device 130 ormerchant camera device captures one or more facial images of the user101, the respective merchant POS device 130 or merchant camera devicegenerates one or more facial templates corresponding to one or more ofthe one or more captured facial images of the user 101. In this exampleembodiment, the merchant POS device 130 or merchant camera devicetransmits the one or more generated facial templates to the accountmanagement system 160 over the network 140. In an example embodiment,the account management system 160 deletes the received facial image. Forexample, the account management system 160 only uses a facial templatecomprising a computer code representation of the facial image of theuser 101. For example, the facial template is of a predetermined size,for example, a 128-byte facial template. In an example, the accountmanagement system 160 generates a facial template comprising a computercode representation of the digital facial image. For example, the facialtemplate may describe key features of the facial image of the user 101,such as shape, color, line, value, space, form, texture, or other usefulor relevant feature of the image or of particular regions of the image.In an example, the facial template is generated by processing the facialimage through a convolutional neural network.

In block 1540, the account management system 160 compares the generatedfacial template against an existing facial template associated with theuser 101 account. In an example embodiment, the account managementsystem 160 receives a user 101 account identifier from the merchant POSdevice 130 that is associated with the user 101 account and identifiesthe user 101 account based on this user 101 account identifier. In thisexample embodiment, the account management system 160 extracts theexisting facial template from the user 101 account associated with thereceived user 101 account identifier. In an example embodiment, theaccount management system 160 compares like features from the generatedfacial template against corresponding like features from the existingfacial template associated with the user 101 account. For example, theaccount management system 160 may compare the key features of thegenerated facial template, such as shape, color, line, value, space,form, texture, or other useful or relevant feature of the image or ofparticular regions of the image against similar key features of theexisting facial template associated with the user 101 account. In anexample embodiment, for each key feature, the account management system160 generates an individual delta value that describes a degree orproportion of difference between the generated facial template and theexisting facial template. In this example embodiment, the accountmanagement system 160 generates an overall delta value or aggregateddelta value based on each individual delta value. For example, theoverall delta value or aggregated delta value gives an overall measureof the degree of difference between the generated facial template andthe existing facial template. In this example, an individual delta valueor overall (or aggregated) delta value may comprise a number between 0and 1, with 1 meaning that the facial templates are completely alike and0 meaning that the facial templates are completely unalike.

In block 1545, the account management system 160 determines whether thedifference between the generated facial template and the existing facialtemplate associated with the user 101 account is greater than athreshold amount. In another example embodiment, the account managementsystem 160 determines whether the difference between the generated audiotemplate and the existing audio template associated with the user 101account is greater than a threshold amount. In an example embodiment,predefining a higher threshold delta amount results in more securityfore users 101 because it makes it more difficult for an illegitimateuser 101 to attempt to access, replace and/or edit user account data.However, predefining a higher threshold delta amount for comparingexisting facial or audio templates against generated facial or audiotemplates may result in inconvenience for the user 101 when the user 101desires to update user account data. For example, a user 101 capturingan image in poor lighting conditions or capturing a sound recording withbackground noise interference may result in an overall or aggregateddelta value that does not exceed the predefined threshold even thoughindividual delta values for other applicable key features may exceed arequired threshold for individual feature-specific delta values. In thisexample embodiment, predefining a lower threshold delta amount resultsin less security for users 101 because it makes it more difficult for anillegitimate user 101 to attempt to access and/or edit user 101 accountdata. However, predefining a lower threshold delta amount for comparingexisting facial or audio templates against generated facial or audiotemplates also may result in more convenience for users 101.

If the account management system 160 determines that the differencebetween the generated facial template and existing facial templateassociated with the user 101 account or existing audio templateassociated with the user 101 account is greater than a thresholddifference, the method 295 proceeds to block 1550. For example, thepredefined overall or aggregated threshold delta value for comparing thedifference of a generated facial template against an existing facialtemplate currently associated with a user 101 account is 0.1 and thedetermined overall or aggregated delta value based on the accountmanagement system 160 comparing the generated facial image against theexisting facial template associated with the user 101 account is 0.12.In certain other example embodiments, instead of determining a overalldelta value that is greater than a predetermined threshold delta valuerepresenting a degree of difference between the generated facial oraudio template and the existing facial or audio template, the accountmanagement system 160 determines an overall delta value that is lessthan a predetermined threshold delta value representing a degree oflikeness between the generated facial or audio template and the existingfacial or audio template.

In block 1550, the account management system 160 transmits anotification to the merchant POS device 130 that the generated facialtemplate does not match the facial template associated with the user 101account. In another example embodiment, the account management system160 transmits a notification to the user computing device 110 that theuser 101 may not update user 101 account data. For example, in responseto determining an overall delta value that is greater than apredetermined threshold delta value representing a degree of differenceor an overall delta value that is less than a predetermined thresholddelta value representing a degree of likeness between the generatedfacial or audio template and the existing facial or audio template, theaccount management system 160 transmits the notification to the usercomputing device 110 that the facial template does not match the facialtemplate associated with the user 101 account and/or that the user 101may not update user 101 account data.

In block 1560, the application 113 displays an error message via theuser computing device 110. For example, the application 113 displays,via the user interface 115 of the user computing device 110, a messagethat reads “User account facial template verification attemptunsuccessful. User may not update account data at this time. Please tryagain later.”

From block 1560, the method 295 returns to block 1510, and the user 101attempts a second or subsequent time to update the user 101 accountdata. In an example embodiment, if the user 101 (or an illegitimateuser) unsuccessfully attempts to update the user 101 account data morethan a predetermined number of times, the account management system 160may otherwise alert the user 101. In other example embodiments, if theuser 101 (or an illegitimate user) unsuccessfully attempts to update theuser 101 account data more than a predetermined number of times, theaccount management system 160 may disable or deactivate the user accountso that the user 101 account may not be accessed or used in anytransactions until the user 101 contacts the account management system160.

Returning to block 1545, in FIG. 15, if the account management system160 determines that the difference between the generated facial templateand existing facial template associated with the user 101 account isless than or equal to a threshold difference, the method 295 proceeds toblock 1570. For example, the predefined overall or aggregated thresholddelta value for comparing the difference of a generated facial templateagainst an existing facial template currently associated with a user 101account is 0.1 and the determined overall or aggregated delta valuebased on the account management system 160 comparing the generatedfacial image against the existing facial template associated with theuser 101 account is 0.05. In certain other example embodiments, insteadof determining a overall delta value that is less than or equal to thana predetermined threshold delta value representing a degree ofdifference between the generated facial or audio template and theexisting facial or audio template, the account management system 160determines an overall delta value that is greater than or equal to apredetermined threshold delta value representing a degree of likenessbetween the generated facial or audio template and the existing facialor audio template.

In block 1570, the account management system 160 transmits anotification to the merchant POS device 130 that the generated facialtemplate matches the facial template associated with the user 101account. For example, the account management system 160 transmits thenotification that the generated facial template matches the facialtemplate associated with the user 101 account to the merchant POS device130 via the network 140. In another example embodiment, the accountmanagement system 160 transmits a notification, via the network 140, tothe merchant POS device 130 that the user 101 may update user 101account data. In yet another example embodiment, the account managementsystem 160 transmits a notification, via the network 140, to themerchant POS device 130 indicating both that the user 101 may updateuser 101 account data and that the generated facial template matches thefacial template associated with the user 101 account.

In block 1580, the merchant POS device 130 displays a request to receivean input of updated user 101 account data. In an example embodiment, themerchant POS device 130 may display, via the user interface of themerchant POS device 130, that the image verification process wassuccessful and requesting that the user 101 provide updated user 101account data. For example, the merchant POS device 130 displays, to theuser 101 and/or the merchant POS device 130 operator, a request thatstates “user 101 image verification successful; please input updateduser 101 account information.”

In block 1590, the merchant POS device 130 receives an input of updateduser 101 account data. In an example embodiment, the user interfacedisplayed by the merchant POS device 130 comprises one or more fieldswherein the user 101 and/or the merchant POS device 130 operator mayprovide one or more inputs comprising updated user 101 account data. Forexample, the user 101 inputs the updated user 101 account data directlyvia the merchant POS device 130 and/or the merchant POS device 130operator inputs the updated user 101 account data to the merchant POSdevice 130 on behalf of the user 101 and at the direction of the user101. For example, via the user interface display of the merchant POSdevice 130, the user 101 and/or the merchant POS device operator mayadd, delete, edit, or rearrange one or more data associated with theuser 101 account. In an example, the merchant POS device operator or theuser 101 may actuate a user interface object on the merchant POS deviceto cause the merchant POS device 130 to submit, via the network 140, theupdated user 101 account data to the account management system 160.

In block 1595, account management system 160 receives the updated user101 account data from the merchant POS device 130 and associates theupdated user 101 account data with the user 101 account. For example,merchant POS device 130 transmits the updated user 101 account data tothe account management system via the network 140 in response toreceiving an input from the user 101 or the merchant POS device 130operator and the account management system 160 receives the updated user101 account data from the merchant POS device 160 via the network 140.In an example embodiment, associating the updated user 101 account datawith the user 101 account comprises deleting all existing user 101account data and replacing it with the received updated user 101 accountdata. In another example embodiment, associating the updated user 101account data with the user 101 account comprises comparing the receivedupdated user 101 account data with existing user 101 account data andoverwriting any existing user 101 account data that does not match thereceived user 101 account data so that the existing user account 101data is edited to match the received user 101 account data. In anexample, the account management system 160 may limit an update of user101 account information to individual fields or categories of user 101account information or may update all user 101 account information.

In certain example embodiments, from block 1595, the method 295 returnsto block 230 in FIG. 2. For example, the user 101 may now engage intransactions according to the methods described herein using the updateduser 101 account data.

Other Example Embodiments

FIG. 16 depicts a computing machine 2000 and a module 2050 in accordancewith certain example embodiments. The computing machine 2000 maycorrespond to any of the various computers, servers, mobile devices,embedded systems, or computing systems presented herein. The module 2050may comprise one or more hardware or software elements configured tofacilitate the computing machine 2000 in performing the various methodsand processing functions presented herein. The computing machine 2000may include various internal or attached components such as a processor2010, system bus 2020, system memory 2030, storage media 2040,input/output interface 2060, and a network interface 2070 forcommunicating with a network 2080.

The computing machine 2000 may be implemented as a conventional computersystem, an embedded controller, a laptop, a server, a mobile device, asmartphone, a set-top box, a kiosk, a vehicular information system, onemore processors associated with a television, a customized machine, anyother hardware platform, or any combination or multiplicity thereof. Thecomputing machine 2000 may be a distributed system configured tofunction using multiple computing machines interconnected via a datanetwork or bus system.

The processor 2010 may be configured to execute code or instructions toperform the operations and functionality described herein, managerequest flow and address mappings, and to perform calculations andgenerate commands. The processor 2010 may be configured to monitor andcontrol the operation of the components in the computing machine 2000.The processor 2010 may be a general purpose processor, a processor core,a multiprocessor, a reconfigurable processor, a microcontroller, adigital signal processor (“DSP”), an application specific integratedcircuit (“ASIC”), a graphics processing unit (“GPU”), a fieldprogrammable gate array (“FPGA”), a programmable logic device (“PLD”), acontroller, a state machine, gated logic, discrete hardware components,any other processing unit, or any combination or multiplicity thereof.The processor 2010 may be a single processing unit, multiple processingunits, a single processing core, multiple processing cores, specialpurpose processing cores, co-processors, or any combination thereof.According to certain embodiments, the processor 2010 along with othercomponents of the computing machine 2000 may be a virtualized computingmachine executing within one or more other computing machines.

The system memory 2030 may include non-volatile memories such asread-only memory (“ROM”), programmable read-only memory (“PROM”),erasable programmable read-only memory (“EPROM”), flash memory, or anyother device capable of storing program instructions or data with orwithout applied power. The system memory 2030 may also include volatilememories such as random access memory (“RAM”), static random accessmemory (“SRAM”), dynamic random access memory (“DRAM”), and synchronousdynamic random access memory (“SDRAM”). Other types of RAM also may beused to implement the system memory 2030. The system memory 2030 may beimplemented using a single memory module or multiple memory modules.While the system memory 2030 is depicted as being part of the computingmachine 2000, one skilled in the art will recognize that the systemmemory 2030 may be separate from the computing machine 2000 withoutdeparting from the scope of the subject technology. It should also beappreciated that the system memory 2030 may include, or operate inconjunction with, a non-volatile storage device such as the storagemedia 2040.

The storage media 2040 may include a hard disk, a floppy disk, a compactdisc read only memory (“CD-ROM”), a digital versatile disc (“DVD”), aBlu-ray disc, a magnetic tape, a flash memory, other non-volatile memorydevice, a solid state drive (“SSD”), any magnetic storage device, anyoptical storage device, any electrical storage device, any semiconductorstorage device, any physical-based storage device, any other datastorage device, or any combination or multiplicity thereof. The storagemedia 2040 may store one or more operating systems, application programsand program modules such as module 2050, data, or any other information.The storage media 2040 may be part of, or connected to, the computingmachine 2000. The storage media 2040 may also be part of one or moreother computing machines that are in communication with the computingmachine 2000 such as servers, database servers, cloud storage, networkattached storage, and so forth.

The module 2050 may comprise one or more hardware or software elementsconfigured to facilitate the computing machine 2000 with performing thevarious methods and processing functions presented herein. The module2050 may include one or more sequences of instructions stored assoftware or firmware in association with the system memory 2030, thestorage media 2040, or both. The storage media 2040 may thereforerepresent examples of machine or computer readable media on whichinstructions or code may be stored for execution by the processor 2010.Machine or computer readable media may generally refer to any medium ormedia used to provide instructions to the processor 2010. Such machineor computer readable media associated with the module 2050 may comprisea computer software product. It should be appreciated that a computersoftware product comprising the module 2050 may also be associated withone or more processes or methods for delivering the module 2050 to thecomputing machine 2000 via the network 2080, any signal-bearing medium,or any other communication or delivery technology. The module 2050 mayalso comprise hardware circuits or information for configuring hardwarecircuits such as microcode or configuration information for an FPGA orother PLD.

The input/output (“I/O”) interface 2060 may be configured to couple toone or more external devices, to receive data from the one or moreexternal devices, and to send data to the one or more external devices.Such external devices along with the various internal devices may alsobe known as peripheral devices. The I/O interface 2060 may include bothelectrical and physical connections for operably coupling the variousperipheral devices to the computing machine 2000 or the processor 2010.The I/O interface 2060 may be configured to communicate data, addresses,and control signals between the peripheral devices, the computingmachine 2000, or the processor 2010. The I/O interface 2060 may beconfigured to implement any standard interface, such as small computersystem interface (“SCSI”), serial-attached SCSI (“SAS”), fiber channel,peripheral component interconnect (“PCI”), PCI express (PCIe), serialbus, parallel bus, advanced technology attached (“ATA”), serial ATA(“SATA”), universal serial bus (“USB”), Thunderbolt, FireWire, variousvideo buses, and the like. The I/O interface 2060 may be configured toimplement only one interface or bus technology. Alternatively, the I/Ointerface 2060 may be configured to implement multiple interfaces or bustechnologies. The I/O interface 2060 may be configured as part of, allof, or to operate in conjunction with, the system bus 2020. The I/Ointerface 2060 may include one or more buffers for bufferingtransmissions between one or more external devices, internal devices,the computing machine 2000, or the processor 2010.

The I/O interface 2060 may couple the computing machine 2000 to variousinput devices including mice, touch-screens, scanners, electronicdigitizers, sensors, receivers, touchpads, trackballs, cameras,microphones, keyboards, any other pointing devices, or any combinationsthereof. The I/O interface 2060 may couple the computing machine 2000 tovarious output devices including video displays, speakers, printers,projectors, tactile feedback devices, automation control, roboticcomponents, actuators, motors, fans, solenoids, valves, pumps,transmitters, signal emitters, lights, and so forth.

The computing machine 2000 may operate in a networked environment usinglogical connections through the network interface 2070 to one or moreother systems or computing machines across the network 2080. The network2080 may include wide area networks (WAN), local area networks (LAN),intranets, the Internet, wireless access networks, wired networks,mobile networks, telephone networks, optical networks, or combinationsthereof. The network 2080 may be packet switched, circuit switched, ofany topology, and may use any communication protocol. Communicationlinks within the network 2080 may involve various digital or an analogcommunication media such as fiber optic cables, free-space optics,waveguides, electrical conductors, wireless links, antennas,radio-frequency communications, and so forth.

The processor 2010 may be connected to the other elements of thecomputing machine 2000 or the various peripherals discussed hereinthrough the system bus 2020. It should be appreciated that the systembus 2020 may be within the processor 2010, outside the processor 2010,or both. According to some embodiments, any of the processor 2010, theother elements of the computing machine 2000, or the various peripheralsdiscussed herein may be integrated into a single device such as a systemon chip (“SOC”), system on package (“SOP”), or ASIC device.

In situations in which the systems discussed here collect personalinformation about users, or may make use of personal information, theusers may be provided with an opportunity or option to control whetherprograms or features collect user information (e.g., information about auser's social network, social actions or activities, profession, auser's preferences, or a user's current location), or to control whetherand/or how to receive content from the content server that may be morerelevant to the user. In addition, certain data may be treated in one ormore ways before it is stored or used, so that personally identifiableinformation is removed. For example, a user's identity may be treated sothat no personally identifiable information can be determined for theuser, or a user's geographic location may be generalized where locationinformation is obtained (such as to a city, ZIP code, or state level),so that a particular location of a user cannot be determined. Thus, theuser may have control over how information is collected about the userand used by a content server.

Embodiments may comprise a computer program that embodies the functionsdescribed and illustrated herein, wherein the computer program isimplemented in a computer system that comprises instructions stored in amachine-readable medium and a processor that executes the instructions.However, it should be apparent that there could be many different waysof implementing embodiments in computer programming, and the embodimentsshould not be construed as limited to any one set of computer programinstructions. Further, a skilled programmer would be able to write sucha computer program to implement an embodiment of the disclosedembodiments based on the appended flow charts and associated descriptionin the application text. Therefore, disclosure of a particular set ofprogram code instructions is not considered necessary for an adequateunderstanding of how to make and use embodiments. Further, those skilledin the art will appreciate that one or more aspects of embodimentsdescribed herein may be performed by hardware, software, or acombination thereof, as may be embodied in one or more computingsystems. Moreover, any reference to an act being performed by a computershould not be construed as being performed by a single computer as morethan one computer may perform the act.

The example embodiments described herein can be used with computerhardware and software that perform the methods and processing functionsdescribed herein. The systems, methods, and procedures described hereincan be embodied in a programmable computer, computer-executablesoftware, or digital circuitry. The software can be stored oncomputer-readable media. For example, computer-readable media caninclude a floppy disk, RAM, ROM, hard disk, removable media, flashmemory, memory stick, optical media, magneto-optical media, CD-ROM, etc.Digital circuitry can include integrated circuits, gate arrays, buildingblock logic, field programmable gate arrays (FPGA), etc.

The example systems, methods, and acts described in the embodimentspresented previously are illustrative, and, in alternative embodiments,certain acts can be performed in a different order, in parallel with oneanother, omitted entirely, and/or combined between different exampleembodiments, and/or certain additional acts can be performed, withoutdeparting from the scope and spirit of various embodiments. Accordingly,such alternative embodiments are included in the scope of the followingclaims, which are to be accorded the broadest interpretation so as toencompass such alternate embodiments.

Although specific embodiments have been described above in detail, thedescription is merely for purposes of illustration. It should beappreciated, therefore, that many aspects described above are notintended as required or essential elements unless explicitly statedotherwise. Modifications of, and equivalent components or actscorresponding to, the disclosed aspects of the example embodiments, inaddition to those described above, can be made by a person of ordinaryskill in the art, having the benefit of the present disclosure, withoutdeparting from the spirit and scope of embodiments defined in thefollowing claims, the scope of which is to be accorded the broadestinterpretation so as to encompass such modifications and equivalentstructures.

What is claimed is:
 1. A computer-implemented method to update useraccount information using image verification, comprising, by one or morecomputing devices: receiving, from a user computing device, an accountidentifier corresponding to a user account associated with a userassociated with the user computing device and a beacon device identifierreceived by the user computing device via a network front a beacondevice; retrieving a stored facial template associated with the useraccount from a plurality of stored facial templates for a plurality ofusers; adding the retrieved facial template to a log of one or morefacial templates corresponding to a plurality of user computing devicesfrom which the beacon device identifier has been previously received,the log including less than all of the plurality of stored facialtemplates for the plurality of users; transmitting, to a servicecomputing device, the log comprising the retrieved facial template, theservice computing device identifying the user account by determiningthat a value of similarity between an image template generated from acaptured image of the user and the transmitted facial template isgreater than a predetermined value of similarity, wherein the servicecomputing device determines the value of similarity by comparing, to thecaptured image, the one or more facial templates in the logcorresponding to the previously received beacon identifier, the one ormore facial templates being advantageously less than a count of theplurality of stored facial templates; receiving, from the servicecomputing device, updated account data and an update request in responseto the service computing device determining that the value of similarityis greater than the predetermined value; and replacing, in the useraccount, one or more items of existing account data with the receivedupdated account data in response to receiving the updated account dataand the update request when the value is greater than the predeterminedvalue.
 2. The computer-implemented method of claim 1, wherein theservice computing device captures the captured image via a cameradevice.
 3. The computer-implemented method of claim 1, wherein the usercomputing device captures the captured image.
 4. Thecomputer-implemented method of claim 1, wherein the captured imagecomprises an image of the user.
 5. The computer-implemented method ofclaim 4, wherein the image of the user comprises an image of a face ofthe user.
 6. The computer-implemented method of claim 1, wherein theupdated account data comprises one or more of updated user contactinformation, updated user demographic information, updated user paymentaccount information, and an updated image template.
 7. Thecomputer-implemented method of claim 1, wherein the updated account datacomprises additions, deletions, or edits to the existing account data.8. The computer-implemented method of claim 1, wherein the generatedimage template comprises a computer code representation of the capturedimage.
 9. The computer-implemented method of claim 1, wherein thecomparing comprises, by the service computing device: determining asimilarity value for one or more regions of the generated template andone or more regions of the transmitted facial template for each of oneor more key features, each similarity value describing a particularsimilarity value between the image template generated based on thecaptured image and the transmitted facial template for the each of theone or more key features; and determining the determined value ofsimilarity between the image template generated based on the capturedimage and the transmitted facial template based on each particularsimilarity value determined for each of the one or more key features.10. The computer-implemented method of claim 9, wherein the one or morekey features comprise one or more of shape, color, line, value, space,form, and texture.
 11. A system to enable updates to user accountinformation in response to facial image verification of users located atservice system locations, comprising: a storage device; and a processorcommunicatively coupled to the storage device, wherein the processorexecutes application code instructions that are stored in the storagedevice to cause the system to: receive, from a user computing device, anaccount identifier corresponding to a user account associated with auser associated with the user computing device and a beacon deviceidentifier received by the user computing device via a wireless network;retrieve, from the user account, a stored facial template associatedwith the user account from a plurality of stored facial templates for aplurality of users; add the retrieved facial template to a log of one ormore facial templates corresponding to a plurality of user computingdevices from which the beacon device identifier has been previouslyreceived, the log including less than all of the plurality of storedfacial templates for the plurality of users; transmit, to a servicecomputing device, the log comprising the retrieved image template, theservice computing device identifying the user account by determiningthat a determined value of similarity between an image templategenerated based on a captured image of the user and the transmittedfacial template is greater than a predetermined value of similarity,wherein the service computing device determines the value of similarityby comparing, to the captured image, the one or more facial templates inthe log corresponding to the previously received beacon identifier, theone or more facial templates being advantageously less than a count ofthe plurality of stored facial templates; receive, from the servicecomputing device, updated account data and an update request in responseto the service computing device determining that the value of similarityis greater than the predetermined value; and replace, in the useraccount, one or more items of existing account data with the receivedupdated account data in response to receiving the updated account dataand the update request when the value is greater than the predeterminedvalue.
 12. The system of claim 11, wherein the image template comprisesa computer code representation of the captured image.
 13. The system ofclaim 11, wherein the updated account data comprises one or more ofadditions, deletions, or edits to the existing account data.
 14. Thesystem of claim 11, wherein the comparing comprises: determining, by theservice computing device, a similarity value for one or more regions ofthe generated template and one or more regions of the transmitted facialtemplate for each of one or more key features, each similarity valuedescribing a particular similarity value between the image templategenerated based on the captured image and the transmitted facialtemplate for the each of the one or more key features; and determining,by the service computing device, the determined value of similaritybetween the image template generated based on the captured image and thetransmitted facial template based on each particular similarity valuedetermined for each of the one or more key features.
 15. The system ofclaim 14, wherein the one or more key features comprise one or more ofshape, color, line, value, space, form, and texture.
 16. The system ofclaim 11, wherein the service computing device captures the capturedimage via a camera.
 17. The system of claim 11, wherein the usercomputing device captures the captured image via a camera.
 18. Thesystem of claim 11, wherein the captured image comprises an image of theuser.
 19. The system of claim 11, wherein the image of the usercomprises an image of a face of the user.
 20. The system of claim 11,wherein the updated account data comprises one or more of updated usercontact information, updated user demographic information, updated userpayment account information, and an updated image template.